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Type  I  error  rates  and  power  of  11  procedures  for  making 
planned  pairwise  multiple  comparisons  of  the  repeated  measures 
means  in  a  split  plot  design  were  investigated.  Using 
simulated  data,  Type  I  error  rates  for  completely  true  null 
hypotheses  and  partially  true  null  hypotheses  were  estimated. 
In  addition,  any-pair  power,  average  per-pair  power,  and  all- 
pairs  power  were  estimated. 

Let  fi^  denote  the  mean  vector  in  the  jth  level  of  a 
between-subjects  factor.  The  11  procedures  were  designed  to 
make  pairwise  comparisons  of  the  elements  of  E/ij/J.  In  each 
procedure  a  Welch-type  test  statistic,  designed  for  use  with 
heterogeneous  variance-covariance  (v-c)  matrices,  was  used. 
Three  types  of  procedures  were  investigated:  Bonferroni 
procedures,  modified  sequentially  rejective  Bonferroni 
procedures,  and  Welsch  step-up  procedures.    Within  each  type, 

V 


procedures  using  trirained  data  and  procedures  using  untriimned 
data  were  investigated. 

The  effects  of  six  factors  were  investigated  in  the 
study:  (a)  distribution  of  the  generated  data,  (b)  sphericity 
of  the  common  v-c  matrix,  (c)  degree  of  the  heterogeneity  of 
the  v-c  matrices,  (d)  total  sample  size,  (e)  sample  size 
arrangement,  and  (e)  configuration  of  means. 

All  11  procedures  controlled  Type  I  error  rates 
reasonably  well.  However,  when  the  design  was  unbalanced  and 
v-c  matrices  were  heterogeneous,  the  Welsch  tests  on  untrimmed 
data  had  a  slight  liberal  tendency.  Two  tests  emerged  as  the 
most  powerful  tests:  the  Welsch  test  on  untrimmed  data  and  the 
Welsch  test  on  trimmed  data.  When  the  data  were  normal  or 
slightly  leptokurtic,  the  Welsch  test  on  untrimmed  data  was 
the  most  powerful  of  the  eleven  tests.  This  was  true  for  any- 
pair  power,  average  per-pair  power,  and  all-pairs  power.  When 
the  data  were  strongly  leptokurtic,  the  Welsch  test  on  trimmed 
data  was  the  most  powerful  test.  Again  this  was  true  for  all 
three  power  criteria. 
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CHAPTER  1 
INTRODUCTION 


A  repeated  measures  design  consists  of  repeated 
observations  Y-,^  on  the  same  dependent  variable  for  every 
subject  i  =  1,  n.     In  this  dissertation,   the  design  of 

interest  is  one  in  which  the  repeated  measures  comprise  a 
single  within-sub jects  factor  with  levels  k  =  1,  K.  Let 

y  denote  the  (fC  x  1)  random  vector  that  generates  the  y^,^. 
Observations  on  y  are  assumed  to  be  independently  and 
identically  distributed  with  mean  vector  ji  and  variance- 
covariance  matrix  E.  If  there  is  a  between-subjects  factor, 
the  design  is  often  called  a  split  plot  design  and  the 
observations  Y-^,^   for  subject  i  =  1,  n^  measured  at  levels 

k  =  1,  .  .  . ,  K  of  the  within-subjects  factor  are  assumed  to  be 
independent  and  identically  distributed  within  each  level  (j 
=  1,  J)  of  the  between-subjects  factor  with  common  mean 

vector        and  variance-covariance  matrix  E ■ . 

Under  normality,  the  validity  conditions  for  the 
univariate  within-subjects  F  tests  have  been  investigated  by 
Huynh  and  Feldt  (1970)  for  the  split  plot  design  with  one 
between-subjects  factor.  They  showed  that  conditions  on  the 
Ej  that  are  necessary  and  sufficient  for  valid  F  tests  for  the 
within-subjects  effects  are  (a)  homogeneity  of  the  variance- 
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covariance  (v-c)  matrices,  for  a  suitable  set  of 
orthonorraalized  variables,  at  all  levels  of  the  between- 
subjects  factor,  and  (b)  sphericity  of  the  common  v-c  matrix. 
These  conditions  have  been  referred  to  as  multisample 
sphericity.  A  v-c  matrix  is  spherical  if  the  variances  for 
all  possible  difference  scores  are  equal.  In  the  repeated 
measures  design  a  necessary  and  sufficient  condition  for  a 
valid  test  of  the  within-subjects  effect  is  that  E  is 
spherical.  Results  in  Box  (1954)  can  be  used  to  show  that 
under  violations  of  sphericity  the  F  test  of  the  within- 
subjects  effect  in  a  repeated  measures  design  is  distributed 
approximately  as  F  with  e{K  -  1)  and  t{n  -  1)(K  -  1)  degrees 
of  freedom.  The  correction  factor  e  has  a  lower  bound  of 
l/(fC  -  1)  and  an  upper  bound  of  one,  which  is  obtained  when 
sphericity  holds.  Similarly,  results  in  Greenhouse  and 
Geisser  (1959)  can  be  used  to  show  that  the  within-subjects 
effects  in  a  split  plot  design  are  distributed  approximately 
as  Fg(,^_^)^  c(N-j)(K-i)  when  the  v-c  matrices  are  homogeneous  but 
not  spherical. 

In  the  repeated  measures  design,  the  t  statistic  for 
testing  a  contrast  on  the  levels  of  the  within-subjects  factor 
is 


t  = 


^  c 2  (1) 


n 


where  MSg.g  denotes  the  error  mean  square  from  the  analysis  of 
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variance.  For  a  split  plot  design  the  t  statistic  for  testing 
a  contrast  on  the  elements  of  }i  -  N"^Snj/ij  is 


2 


(2) 


N 

where  MSg.g^^  is  the  error  mean  square  for  the  within-subjects 
effects,  y.,^  is  the  Ath  element  of  y  =  N'''Snjyj,  and  is  the 
sample  mean  vector  for  the  jth  group.  Using  MSg.g  in  equation 
(1)  is  based  on  the  sphericity  assumption  in  the  repeated 
measures  design;  using  MSg.g/^  in  equation  (2)  is  based  on  the 
multisample  sphericity  assumption  in  the  split  plot  design. 

Boik  (1981)  investigated  the  effects  of  nonsphericity  on 
the  Type  I  error  and  power  rates  of  a  test  using  t  given  in 
equation  (2)  and  the  critical  value  tt^,^,  (n-j)(k-1)-  found 
that  for  very  small  departures  from  sphericity,  the  test  size 
can  be  quite  discrepant  from  a.  For  .5  <  e  <  1.0,  increasing 
the  sample  size  increases  the  bias  of  the  test.  Boik 
recommended    employing  the  statistic 


t  = 


c'Sc 

\-N- 


(3) 


where  c  is  a  contrast  vector  constructed  from  the  C|^  and  S  is 
the  pooled  v-c  matrix  and  it^^j,    n-j  the  critical  value. 

Boik  showed  that  the  power  of  the  test  based  on  equation  (2) 
can  be  larger  or  smaller  than  the  power  of  the  test  based  on 
equation  ( 3 ) . 
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The  data  in  repeated  measures  designs  and  split  plot 
designs  seldom  satisfy  the  sphericity  assumption.  In 
addition,  the  data  in  a  split  plot  design  may  have 
heterogeneous  v-c  matrices.  Several  authors  have  investigated 
Type  I  error  rates  for  multiple  comparison  procedures  applied 
to  levels  of  the  within-subjects  factor  in  the  repeated 
measures  design  and  in  the  split  plot  design. 

In  a  Monte  Carlo  study,  Maxwell  (1980)  investigated  the 
impact  of  violating  sphericity  on  Type  I  error  rates  for 
pairwise  comparisons  in  a  repeated  measures  design.  He 
investigated  a  procedure  in  which  eguation  (1)  and  Tukey's 
critical  value  is  used  and  showed  that  the  procedure  results 
in  actual  Type  I  error  rates  above  the  nominal  level,  with  the 
discrepancy  increasing  as  e  declines.  Maxwell  also 
investigated  procedures  in  which  (c '  Sc/n)''''^  replaces  the 
denominator  of  equation  (1),  and  concluded  that  this  statistic 
with  the  Bonferroni  critical  value  results  in  the  best  control 
of  the  familywise  Type  I  error  rate. 

Keselman,  Keselman,  and  Shaffer  (1991)  investigated  four 
procedures  that  can  be  used  to  make  multiple  pairwise 
comparisons  of  the  elements  of  the  vector  Ji*  =  J'''  Ifi.  in  a 
split  plot  design.  For  their  test  statistic,  they  replaced 
the  denominator  of  equation  (2)  by  [I  ■  C  S^c/n^]'^^^  where  c  is 
the  K  X  1  vector  of  contrast  weights.  They  included  the 
following  critical  values:  Studentized  maximum  modulus, 
Bonferroni,    Studentized   range,    and   an   analog   of  Cochran's 


(1964)  critical  value,  each  with  Satterthwaite  degrees  of 
freedom.  Keselman,  Keselraan,  and  Shaffer  concluded  that  their 
test  statistic  with  a  Studentized  range,  a  Studentized  maximum 
modulus,  or  a  Bonferroni  critical  value  provides  reasonable 
control  over  the  Type  I  error  rates.  The  Bonferroni  critical 
value  resulted  in  the  smallest  Type  I  error  rate  and  the 
Studentized  maximum  modulus  critical  value  resulted  in  a  more 
powerful  test  than  did  the  Bonferroni  critical  value. 
Keselman,  Keselman,  and  Shaffer  (1991)  found  that  Type  I  error 
rates  decreased  as  e  decreased  regardless  of  the  level  of 
heterogeneity. 

Keselraan  (1994)  investigated  Type  I  error  rates  and  power 
for  28  multiple  comparison  procedures.  Keselman  adapted 
Welsch's  (1977)  step-up  procedure  by  using  the  test  statistic 
from  Keselman,  Keselman,  and  Shaffer  (1991)  with  critical 
values  calculated  as  described  in  Welsch.  They  concluded  that 
Welsch's  step-up  procedure  was  robust  and  tended  to  be  more 
powerful  than  the  other  procedures  included  in  their  study. 

There  has  been  a  limited  amount  of  work  on  the  impact  of 
nonnormality  on  Type  I  error  rates  of  procedures  for  testing 
multiple  comparisons  of  the  levels  of  the  within-subject 
effect.  Keselman,  Keselman,  and  Shaffer  found  that  when  the 
Studentized  range  critical  value  was  used,  there  was  a 
tendency  for  Type  I  error  rate  to  be  larger  when  the  data  were 
sampled  from  chi-sguare  distributions.     There  was  no 


consistent  trend  when  the  Studentized  maximum  modulus  and 
Bonferroni  critical  values  were  used. 

A  few  authors  have  investigated  the  effect  of 
nonnormality  on  the  power  of  the  F-tests  by  theoretical  (David 
&  Johnson,  1951;  Srivastava,  1959;  Tiku,  1971)  and  by 
numerical  approaches  (Donaldson,  1968;  Srivastava,  1959)  in 
the  one-way  independent  samples  design.  Srivastava  (1959) 
found  that  skewness  has  a  small  effect  on  the  power  of  the  F 
test  but  kurtosis  leads  to  a  noticeable  change  in  the  power 
particularly  in  the  case  of  small  samples.  For  example,  when 
the  degrees  of  freedom  are  4  and  10  and  the  population  is 
leptokurtic  (Xj  =  0,  =  2.4),  the  power  of  the  F  test  is 
higher  than  the  normal-theory  power  up  to  the  point  at  which 
power  is  approximately  0.8.  In  the  region  of  very  high  power, 
the  power  of  the  F  test  under  nonnormality  is  smaller  than  the 
normal-theory  power.  The  reverse  happens  when  the  kurtosis  is 
negative.  Donaldson  (1968)  investigated  the  robustness  of 
Type  I  and  II  error  rates  for  the  F  test  applied  to  data 
sampled  from  the  exponential  or  lognormal  distributions.  He 
concluded  that  when  the  test  is  applied  to  nonnormal  data,  it 
is  slightly  less  powerful  than  when  test  is  applied  to  normal 
data.  As  the  sample  size  increases,  the  power  differences 
decline. 

There  has  been  a  limited  amount  of  work  on  the  effects  of 
nonnormality  on  the  power  in  the  repeated  measures  design  or 
the  split  plot  design.     Rogan,   Keselman,   and  Mendoza  (1979) 


investigated  the  effect  of  nonnormality  on  Type  II  error  rates 
of  the  F  test  in  the  split  plot  design  using  data  sampled  from 
chi-square  distributions  with  three  degrees  of  freedom. 
Rogan,  Keselman,  and  Mendoza  found  that  power  rates  for  the 
within-subjects  main  effect  and  for  the  within  x  between 
interaction  were  slightly  lower  for  the  nonnormal 
distribution. 

Yuen  (1975)  compared  the  power  of  the  one-sample 
Winsorized  t,  trimmed  t  and  Student's  t  for  long-tailed 
distributions.  The  trimmed  t  and  Winsorized  t  were  more 
powerful  than  Student's  t  for  sample  sizes  of  10  or  above, 
when  the  amount  of  trimming  was  appropriate.  Yuen  and  Dixon 
(1973)  investigated  the  power  of  two-sample  trimmed  t  test 
under  the  assumption  of  equal  population  variances.  The  power 
of  trimmed  t  never  exceeded  the  power  of  Student ' s  t  under 
normality,  but  with  sample  size  of  greater  than  7  the  loss  of 
power  was  small.  For  contaminated  normal  distributions  and 
sample  sizes  not  less  than  10,  the  power  of  the  trimmed  t  was 
superior  to  that  of  Student's  t.  Yuen  (  1974)  reported  that  in 
the  case  of  unequal  sample  sizes  and  variances  the  power  of 
the  trimmed  Welch  test  is  superior  to  the  power  of  Welch's 
test  for  long-tailed  distributions  and  it  is  comparatively 
more  powerful  if  the  larger  sample  size  is  associated  with  the 
larger  variance  than  with  the  smaller  variance.  Wilcox  (1992) 
investigated  the  effect  of  nonnormality  on  Type  I  error  rate 
and   power   when    comparing   two    independent    groups.  Wilcox 
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showed  that  the  power  of  Welch's  procedure  drops  dramatically 
as  the  amount  of  contamination  gets  large  and  Yuen's  trimmed 
Welch  test  has  more  power  than  Welch's  test. 

Under  the  violation  of  the  multisample  sphericity 
assumption  and/or  normal  distribution  assumption,  Wilcox 
(199  3)  compared  Type  I  error  and  power  rates  of  the  usual  e- 
adjusted  F  test  (F)  and  an  e-adjusted  F  test  using  trimmed 
means  and  Winsorized  variances  (D)  in  repeated  measures 
designs.  Wilcox  concluded  that  Type  I  error  rates  for  D  and 
F  were  in  close  agreement.  Both  procedures  were  too 
conservative  when  n  was  small  and  e  was  close  to  one.  When  e 
is  relatively  small,  the  F  test  can  have  Type  I  error  rate 
slightly  higher  than  the  nominal  level  when  the  distributions 
are  normal.  In  contrast,  D  has  Type  I  error  rate  slightly 
below  the  nominal  level.  The  power  of  D  is  substantially 
better  than  F  when  the  distributions  have  heavy  tails  and  the 
advantage  in  using  D  increases  for  asymmetric  distributions. 

The  Problem 

There  has  been  a  limited  amount  of  research  on  the 
robustness  or  power  of  pairwise  multiple  comparison  procedures 
when  the  normality  and/or  multisample  sphericity  assumption 
are  violated  in  a  split  plot  design.  However,  results  in 
Keselman,  Keselman,  and  Shaffer  (1991),  Yuen  (1974),  and 
Wilcox  (1992)  suggest  a  Welch-type  test  using  trimmed  means 
and  Winsorized  variances  may  be  both  robust  and  powerful. 
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Purpose  of  the  Study 

The  purpose  of  this  study  is  to  investigate  Type  I  error 
rates  and  power  for  pairwise  multiple  comparison  procedures 
when  the  assumptions  of  the  pairwise  comparison  procedures  are 
violated  in  a  split  plot  design  with  one  between-subjects 
factor. 

Significance  of  the  Study 

Studies  show  that  long-tailed  distributions  are  common  in 
psychometric  and  educational  measures  (Micceri,  1989;  Wilcox, 
1990).  It  is  known  that  long-tailed  distribution  can  reduce 
the  power  of  hypothesis  testing  procedures  based  on  means 
(Tukey,  1960;  Tukey  and  McLaughlin,  1963;  Yuen,  1974). 
Further  tests  based  on  trimmed  means  and  Winsorized  variances 
maintain  power  with  long-tailed  distributions  while  resulting 
in  a  small  power  loss  with  normal  and  short  tailed 
distributions.  Consequently,  the  development  and  evaluation 
of  multiple  comparison  procedures  using  trimmed  means  and 
Winsorized  variances  is  of  interest  in  connection  with  testing 
contrasts  on  levels  of  the  within-subjects  factor. 


CHAPTER  2 
LITERATURE  REVIEW 


Linear  Model  for  the  Repeated  Measures  Design 
with  One  Between-Subjects  Factor 

Consider  a  design  in  which  there  are  J  levels  of  a 
between-subjects  factor  (groups)  with  subjects  in  the  jth 
level.  Each  subject  is  measured  at  each  of  K  levels  of  a 
within-subjects  factor  (treatments  or  trials).  This  design  is 
sometimes  called  a  split  plot  design.  Using  the  traditional 
notation,  the  univariate  linear  model  describing  this  design 
is 

Yi^^   =  P  +  «j  +  3k  +  ^i(j)    +  «3jk  +  3^ki(j)     +  Eijk'  (4) 

i  —  1 ,  .  .  .  ,  Tij  / 
j=l,...,J; 
k—  l/...,i?^^ 

where  Y^j,^  is  the  observed  value  of  subject  i  within  group  j 
at  level  k  of  the  within-subjects  factor,  p  is  the  overall 
mean,  is  the  effect  of  group  j,  [3,^  is  the  effect  of  the 
level  k  of  the  within-subjects  factor,  ;t.(.)  is  the  effect  of 
subject  i  nested  within  group  j,  afB-,^  is  the  interaction  of 
group  J  with  level  k  of  the  repeated  measures  factor,  \in^^^^^^ 
is  the  interaction  of  subject  i  within  group  j  with  level  k  of 
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the  repeated-measures  factor  and  e^j,^  is  a  residual.  The 
observations  (Y-j,jS)  are  assumed  to  be  independently  and 
identically  distributed  within  each  level  j  of  the  between- 
subjects  factor  with  common  mean  vector  ji-  and  v-c  matrix  Ej . 

In  the  split  plot  design,   the  hypothesis  of  no  within- 
subjects  effect  is  tested  by  the  ratio 

SS^ 


(N-J)  {K-D 


and  the  hypothesis  of  no  interaction  effect  is  tested  by  the 
ratio  ',■  '  : 


^^AB 


^^B-S/A  ^^B-S/A 

(N-J)  (K-l)  .  ,  .    K  . 

where  N  is  the  total  number  of  subjects.  The  ratios  are 
referred  to  the  F  distribution  with  degrees  of  freedom  (K-l) 
and  (N  -  J)  (K  -  1),  and  (J  -  1)(K  -  1)  and  {N  -  J){K  -  1), 
respectively. 

Validity  Conditions  for  Univariate  F  Tests 

Huynh  and  Feldt  (1970)  investigated  conditions  on  the  Z- 
required  for  F^  and  to  follow  the  F  distribution  described 
in  the  preceding  paragraph.  They  showed  that  necessary  and 
sufficient  conditions  on  the  E-  for  the  valid  F  tests  of  the 
within-subjects    effects    are     (a)     homogeneity    of    the  v-c 
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matrices  for  a  suitable  set  of  orthonormalized  variables  at 
all  levels  of  the  between-subjects  factor,  and  (b)  sphericity 
of  the  common  v-c  matrix.  These  conditions  have  been  referred 
to  as  raultisample  sphericity.  A  v-c  matrix  is  spherical  if 
the  variances  for  all  possible  difference  scores  are  equal. 
With  the  homogeneity  of  the  v-c  matrices  assumption,  the 
variation  of  the  interaction  of  the  subject  and  the  within- 
subject  factors  is  homogeneous  for  the  groups.  This  allows 
pooling  over  groups  to  calculate  the  sum  of  squares  for  the 
interaction  of  the  within-subjects  factor  and  subjects  within 
groups  (Brogan  &  Kutner,  1980).  If  a  design  has  one  within- 
siibjects  factor  but  no  between-subjects  factors,  the  condition 
on  £  that  is  necessary  and  sufficient  for  a  valid  test  of  the 
within-subjects  effect  is  that  E  is  spherical  (Huynh  &  Feldt, 
1970). 

Effects  of  the  Violations  of  Validity  Conditions 
for  Univariate  F  tests 

Robustness  against  Non-sphericity  ' 

Repeated  observations  seldom  satisfy  the  sphericity 
assumption  (Rogan,  Keselman,  &  Mendoza,  1979).  For  example, 
when  the  within-subjects  factor  is  occasion,  adjacent 
measurements  are  usually  more  highly  correlated  than  non- 
adjacent  measurements  in  split  plot  and  repeated  measurements 
designs  (Huynh  and  Feldt,  1980;  Rogan,  Keselman,  &  Mendoza, 
1979).  Box  (1954),  studying  the  repeated  measures  design,  and 
Imhof  (1962),   studying  the  split  plot  design,   found  that  as 
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the  degree  of  non-sphericity  increased,  F  values  for  tests 
involving  the  within-subjects  factor  inflated. 

Based  on  theory  in  Box  (1954),  it  can  be  shown  that  when 
the  sphericity  assumption  is  violated  in  a  repeated  measures 
design,  the  mean  square  ratio  for  the  within-subjects  factor 
is  approximately  distributed  as  F  with  reduced  degrees  of 
freedom.  Similarly,  theory  in  Geisser  and  Greenhouse  (1958) 
can  be  used  to  show  that  mean  square  ratios  for  the  within- 
subjects  effects  are  approximately  distributed  as  F  with 
reduced  degrees  of  freedoms.  Consequently  under  violations  of 
sphericity,  the  within-subjects  tests  in  the  repeated  measures 
and  split  plot  designs  will  be  liberal.  Several  authors  have 
used  simulation  methods  to  estimate  Type  I  error  rates  when 
sphericity  is  violated  in  the  repeated  measures  design  (Huynh 
&  Feldt,  1976;  Stoloff,  1970)  and  in  the  split  plot  design 
(Collier,  Baker,  Mandeville,  &  Hayes,  1976;  Wilson,  1975). 
These  studies  have  all  demonstrated  increased  Type  I  error 
rates  when  sphericity  is  violated. 

As  noted  above,  theory  in  Box  (1954)  can  be  used  to  show 
that  under  violations  of  sphericity  the  F  test  of  the  within- 
subjects  main  effect  is  distributed  approximately  as  F  with 
e(K  -  1)  and  e(n  -  1){K  -  1)  degrees  of  freedom.  The 
correction  factor  e  has  a  lower  bound  of  1/{K  -  1)  and  an 
upper  bound  of  one,  which  is  obtained  when  sphericity  holds. 
The  correction  factor  e  is  defined  by 
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where  a^^  (t#s;  s=l,  ...  ,K)  is  the  tsth  element  of  the  matrix 
E,  a^^  is  the  mean  of  the  variances,  o^.  is  the  mean  of  the  tth 
row,  and  a. .  is  the  grand  mean  of  all  elements  in  the 
population  v-c  matrix. 

Two  estimates  of  e  are  available.     The  first,  suggested 
by  Greenhouse  and  Geisser  (1959),  is 

e  =   — H —  .  (8) 

The  second  suggested  by  Huynh  and  Feldt  (1976),  is 

g  =  [N{K-l)e-2]  g 

{K-D  [N-J-  iK-l)i]  ■  -     ■        ^  ' 

Lecoutre  (1991)  corrected  a  minor  error  in  the  formula  for  e: 

g  =      {N-J+D  {K-D  i-2 

(K-l)  [N-J-(K-l)i]  ■  ,  ^  ' 

In  a  split  plot  design,  the  degrees  of  freedom  for  the  within- 
subjects  effects  are  approximately  e{K  -  1)  and  e {N  -  J){K  - 
1)  when  sphericity  is  violated.  In  this  design  e  is 
calculated  by  applying  equation  (5)  to  the  pooled  v-c  matrix. 

A  number  of  studies  have  used  simulated  data  to  estimate 
Type  I  error  rates  for  the  repeated  measures  design  (Huynh  & 
Feldt,  1976;  Stoloff,  1970)  and  the  split  plot  design 
(Collier,  Baker,  Mandeville,  &  Hayes,  1976;  Huynh,  1978; 
Keselraan  &  Keselman,   1990;  Maxwell  &  Arvey,   1982;  Rogan, 


Keselraan,  &  Mendoza,  1979,  Wilson,  1975)  when  e  and  e  are  used 
to  adjust   critical   values.      In   general   these   studies  have 
shown  that  the  performance  of  the  tests  depends  on  e .    When  e 
is  near  1.0,  use  of  the  e-adjusted  test  tends  to  result  in  an 
estimated    actual    Type    I    error    rate    (r)    below    a    and  a 
discrepancy    between    r    and    a    that     is     larger    than  the 
discrepancy  for  the  e-adjusted  test.     As  e  declines,  the  e- 
adjusted  test  achieves  better  control  of  z  than  the  e-adjusted 
test.    Rough  guidelines  are  that  the  e-adjusted  test  achieves 
better  control  than  the  e-adjusted  test  for  e  >  .75,  about  the 
same  control  for  .50  <  e  <  .75  and  worse  control  for  e  <  .50. 
According  to  Huynh  and  Feldt,    e    is   typically  above    .75  in 
educational  and  psychological  research. 
Robustness  against  V-C  Heteroscedasticity 

According  to  Keselman  and  Keselraan  (1990),  the  majority 
of  studies  investigating  the  robustness  of  the  univariate 
repeated  measurements  approach  to  departures  from  its 
assumptions  have  focused  on  the  effects  of  non-sphericity  of 
the  v-c  matrix  and  not  on  the  effects  of  heterogeneity  of  the 
v-c  matrices. 

Rogan,  Keselman,  and  Mendoza  (1979)  investigated  the 
effect  of  heterogeneous  v-c  matrices  in  balanced  split  plot 
designs.  They  found  that  e-  and  e-adjusted  univariate 
procedures  are  robust  with  Type  I  error  rates  slightly  above 
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a  for  both  within-subjects  main  effect  and  between  x  within 
interaction. 

Keselman  and  Keselman  (1990)  investigated  the  effect  of 
heterogeneous  v-c  matrices  on  Type  I  error  rate  in  balanced 
and  unbalanced  split  plot  designs  using  three  estimates  of  e 
(e,  e,  and  e  =  min[l,  (e  +  e)  /  2],  Quintana  &  Maxwell,  1985) 
to  adjust  critical  values.  They  tested  the  main  effect 
hypothesis  Hq^  :  jj^  =  jj^  =  •  •  '  =  Py^i  where  is  the  kth 
element  of  /i  =  N"""  In^jij,  N  =  n^  +  nj  +  •  •  •  +  nj  and  is  the 
(K  X  1)  mean  vector  for  the  jth  group.  They  found  that 
heterogeneity  affects  the  size  of  the  test  of  the  between  x 
within  interaction  when  group  sizes  are  unegual,  but  has 
little  effect  on  Type  I  error  rate  of  the  within-subjects  main 
effect  when  group  sizes  are  egual  or  unegual. 

Huynh  (1978)  extended  the  e-  and  e -adjusted  tests  to  the 
General  Approximation  (GA)  test  and  the  Improved  General 
Approximation  (IGA)  test  to  take  the  heteroscedasticity  of  the 
v^c  matrices  into  account.  Huynh  conducted  a  simulation  study 
and  compared  four  approximate  tests:  the  e-adjusted,  e- 
adjusted,  GA,  and  IGA  tests.  Huynh  included  v-c  matrices  that 
violated  multisample  sphericity  and  both  balanced  and 
unbalanced  split  plot  designs.  Huynh  tested  the  main  effect 
hypothesis  tested  in  Keselman  and  Keselman.  Huynh  found  that 
both  the  e  approximate  and  GA  procedures  are  conservative,  and 
the  e  approximate  and  IGA  tests  tend  to  be  liberal  even  though 
these  tests  performed  reasonably  over  a  range  of  situations. 


Huynh  concluded  that  the  IGA  tests  of  the  within-subjects  main 
effect  and  the  between  x  within  interaction  control  the  Type 
I  error  rate  when  the  v-c  matrices  are  heterogeneous. 
However,  the  Type  I  error  rates  for  the  e -adjusted  tests  were 
very  similar  to  those  for  the  IGA  tests.  Huynh  concluded  that 
the  e-adjusted  tests  are  adequate  because  the  IGA  test  is 
computationally  complex.  However,  Algina  and  Oshiraa  (1994) 
have  written  a  SAS  program  to  calculate  the  GA  and  IGA  test. 

Algina  and  Oshima  (1994)  investigated  Type  I  error  rates 
for  the  e-adjusted,  GA,  IGA,  and  a  corrected  IGA  (GIGA)  test 
based  on  Lecoutre ' s  (1991)  results.  They  included  more  severe 
heterogeneity  than  did  Huynh  (1978).  Algina  and  Oshiraa 
reported  that  the  e-adjusted  test  of  the  between  x  within 
interaction  was  not  robust  to  heterogeneity  and  that  the  IGA 
and  GIGA  test  provide  reasonable  control  of  the  Type  I  error 
rate.  The  e-adjusted,  IGA,  and  GIGA  test  provided  adequate 
control  of  the  Type  I  error  rate  for  the  test  of  Hq^    :  Jj^  =  jj^ 

=  •  •  •   =  /^K- 

Algina  and  Oshima  (in  press)  investigated  the  effect  of 
heterogeneous  v-c  matrices  on  the  Type  I  error  rates  of 
adjusted  F  tests  of  the  null  hypothesis  Hqj  :  jj*  =  =  '  •  ' 
=  Jj^  where  ^|^*  is  the  kth  element  of  p*  =  Ejj.  and  ji-  is 
the  mean  vector  for  the  jth  group.  They  included  the  e- 
adjusted  test,  the  GA  test,  the  IGA  test,  and  the  Corrected 
Improved  General  Approximation  (GIGA)  test  based  on  Lecoutre ' s 
(1991)  result.     In  contrast  to  the  Keselman  and  Keselman 
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results  in  which  the  e -adjusted  test  of  Hq^  provided  adequate 
control  of  r,  the  e-adjusted  test  of  Hqj  provided  poor  control 
of  the  Type  I  error  rate.  Algina  and  Oshima  concluded  that 
the  CIGA  test  provides  good  control  over  the  Type  I  error 
rate.  The  IGA  test  also  provided  good  control  over  the  Type 
I  error  rate,  but  is  slightly  more  liberal  than  the  CIGA  test 
when  both  have  Type  I  error  rates  above  a  =  .05  and  slightly 
less  conservative  when  both  have  Type  I  error  rates  below  a  = 
.05. 

Effect  of  Nonnormality  on  Type  I  Error  Rates 

A  number  of  theoretical  (Gayen,  1950;  Tiku,  1964)  and 
numerical  investigations  (Berenson,  1983;  Donaldson,  1968; 
Hack,  1958;  Pearson,  1931)  of  the  effect  of  nonnormality  on 
Type  I  error  rates  in  the  one-way  analysis  of  variance  model 
have  been  made,  but  there  has  been  limited  amount  of  work 
(Algina  &  Oshima,  1994;  Algina  &  Oshima,  in  press;  Wilcox, 
1993)  on  the  impact  of  nonnormality  on  type  I  error  rates  for 
the  repeated  measures  design  and  the  split  plot  design. 

Gayen  (1950)  expressed  the  distribution  of  F  ratio  for 
nonnormal  populations  by  the  first  four  terms  of  Edgeworth's 
series  expansion.  He  showed  that  Type  I  error  rate  is  related 
to  the  skewness  and  kurtosis  of  a  distribution.  Tiku  (1964) 
expanded  probability  density  functions  of  the  variance 
components  in  terras  of  Laguerre  polynoraials.  He  showed  that 
effects  due  to  nonnormality  are  larger  if  the  skewness  is  in 
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different  directions  in  different  groups.  Pearson  (1931) 
studied  the  effect  of  nonnormality  on  Type  I  error  rate  of  the 
F  test  using  empirical  data  with  egual  group  sizes  in  a  one- 
way independent  samples  design.  Pearson  concluded  that  if  the 
deviation  from  normality  is  not  extreme,  the  F  test  is  guite 
robust.  Recently,  Berenson  (1983)  reexamined  the  issue  in  a 
variety  of  situations  when  sample  sizes  are  small  in  a  one-way 
independent  samples  design.  Berenson  found  that  kurtosis  has 
a  much  greater  effect  on  the  Type  I  error  rate  of  the  F  test 
than  does  skewness.  It  was  concluded  that  the  F  test  tends  to 
be  conservative  for  the  heavy-tailed  populations. 

Algina  and  Oshima  (1994)  investigated  the  effect  of 
nonnormality  on  the  Type  I  error  rate  of  the  F  test  using  data 
sampled  from  multivariate  normal  and  multivariate  lognormal 
distributions  in  a  split  plot  design.  The  main  effect  null 
hypothesis  of  the  within-subject  factor  was  Eq^  :  =  ^2  =  '  '  ' 
=  where  Ji  =  N"""  In^pi^  and  ji^  is  the  mean  vector  for  the  jth 
group.  Algina  and  Oshima  included  three  tests  designed  for 
use  when  multisample  sphericity  condition  is  violated:  the  GA 
test,  the  IGA  test,  and  the  Corrected  Improved  General 
Approximation  (GIGA)  test.  Algina  and  Oshima  showed  that, 
when  the  sample  sizes  are  egual  and  v-c  matrices  are 
homogeneous.  Type  I  error  rates  are  uniformly  lower  in  the 
multivariate  lognormal  conditions  than  in  the  multivariate 
normal  conditions  for  interaction  tests.  For  the  main  effect 
tests,   the  effect  of  sampling  from  a  multivariate  lognormal 
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distribution  was  that  Type  I  error  rates  tended  to  be  reduced 
for  e  =  1.00  and  to  be  increased  for  e  =  .40.  The  effect  of 
sampling  from  a  multivariate  lognormal  distribution  on  the 
within-subjects  main  effect  was  complex  for  the  conditions  in 
which  v-c  matrices  were  heterogeneous.  For  K  =  4,  Type  I 
error  rates  for  e  =  1.00  were  reduced  and  those  for  e  =  .40 
were  increased.  For  K  =  8  and  S^rSgrSj  =  1.0:1.5:2.0,  Type  I 
error  rates  for  e  =  1.00  and  .75  were  reduced  and  those  for  e 
=  .40  were  increased.  For  K  =  8  and  I,^:Il2'-^3  =  1:5:9,  the 
Type  I  error  rates  for  e  =  1.00  were  reduced,  those  for  e  = 
.75  tended  to  be  reduced  and  those  for  e  =  .40  tended  to  be 
increased. 

Algina  and  Oshima  (in  press)  investigated  the  effect  of 
nonnormality  on  the  Type  I  error  rates  for  the  F  test  of  the 
main  effect  of  the  within-subjects  factor  in  a  split  plot 
design  under  violations  of  multisample  sphericity.  Algina  and 
Oshima  included  four  tests:  the  e-adjusted  test,  the  GA  test, 
the  IGA  test,  and  the  CIGA  test.  The  main  effect  null 
hypothesis  was  jj*   =  /ij*   =  *  "  '   =  ^k*   where  Ji*  =  J"^  Ifi. 

and  ji-  is  the  mean  vector  for  the  jth  group.  Algina  and 
Oshima  found  that  the  Type  I  error  rate  increases  as  e 
decreases  for  both  data  sampled  from  multivariate  normal  and 
multivariate     lognormal     distributions.  The     effect  of 

nonnormality  depended  on  e.     For  e  >  0.75,  the  Type  I  error 
rates  were  lower  when  sampling  from  a  multivariate  lognormal 
distribution;  for  e  =  .40,  Type  I  error  rates  were  higher. 
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Effect  of  Nonnormality  on  Type  II  Error  Rates 

Investigations  of  the  effect  of  nonnormality  on  power  of 
the  one-way  independent  samples  F  test  have  been  conducted  by 
theoretical  (David  &  Johnson,   1951;   Srivastava,   1959;  Tiku, 
1971)  and  numerical  approaches  (Donaldson,   1968;  Srivastava, 
1959) . 

David  and  Johnson  (1951)  obtained  approximations  of  the 
power  function  of  F  test.  A  numerical  investigation  showed 
that  the  probabilities  with  the  approximation  function  were 
generally  close  to  the  exact  values.  Srivastava  (1959) 
expressed  the  power  function  of  the  F  test  for  moderately 
nonnormal  populations  by  the  first  four  terms  of  an  Edgeworth 
series.  He  assumed  an  equal  number  of  observations  in  the 
samples.  Srivastava  found  that  skewness  had  little  effect  on 
power  of  the  F  test  but  kurtosis  resulted  in  a  noticeable 
change  in  the  power  particularly  in  the  case  of  small  samples. 
For  example,  when  the  degrees  of  freedom  are  4  and  10  and  the 
population  is  leptokurtic  (A3  =  0,  =  2.4),  the  power  of  the 
F  test  is  higher  than  the  normal-theory  power  up  to  the  point 
at  which  power  is  approximately  0.8.  In  the  region  of  very 
high  power,  the  power  of  the  F  test  under  nonnormality  is 
smaller  than  the  normal-theory  power.  The  reverse  happens 
when  the  kurtosis  is  negative.  Tiku  (1971)  expressed  for  the 
power  function  of  the  F  test  with  nonnormal  data  by  first  six 
terras  of  the  Laguerre  series  expansions.  A  simple  two  moment 
approximation   for   near-normal   population   distributions  was 
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also  derived.  Tiku  found  that  the  effect  of  moderate 
nonnormality  on  the  power  of  F  test  is  unimportant,  and  the 
two  moment  approximation  gives  accurate  approximations  in 
cases  of  near-normal  populations.  However,  kurtosis  had  a 
greater  effect  on  the  power  than  skewness.  For  the  condition 
of  his  experiment,  =  4  and  =  20,  the  effect  of  kurtosis 
was  similar  to  one  reported  in  Srivastava.  Donaldson  (1968) 
investigated  the  robustness  of  the  Type  I  and  II  error  rates 
for  the  F  test  applied  to  data  sampled  from  the  exponential  or 
lognormal  distributions.  Donaldson  concluded  that  when  the 
test  is  applied  to  nonnormal  data,  it  is  slightly  less 
powerful  than  when  test  is  applied  to  normal  data.  As  the 
sample  size  increases,  the  power  differences  decline. 

There  has  been  a  limited  amount  of  work  on  the  effects  of 
nonnormality  on  the  power  in  repeated  measures  design  or  split 
plot  design.  Rogan,  Keselman,  and  Mendoza  (1979)  investigated 
the  effect  of  nonnormality  on  Type  II  error  rates  of  the  F 
test  in  the  split  plot  design  using  data  sampled  from  a  chi- 
square  distribution  with  three  degrees  of  freedom.  Rogan, 
Keselman,  and  Mendoza  found  that  under  nonnormality  power 
rates  increase  slightly  for  the  within-subjects  main  effect 
arid  for  the  between  x  within  interaction. 
Use  of  the  Trimmed  t  Test  under  Nonnormality 

The   classical    statistics    such   as   the   sample  mean  and 
variance     are     sensitive     to     outliers.  Trimming  and 

Winsorization  refer  to  the  removal  and  modification  of  the 
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extreme  values  of  a  sample.  The  purpose  of  the  trimming  or 
Winsorization  is  to  reduce  the  sensitivity  to  outliers.  Let 
7(^5  denote  the  ith  order  observation  in  a  random  sample  of 
size  N  with  y(^)  <  y^2)  <  •  •  •  <  Y^f^)•  The  a-trimmed  mean  for 
a  =  g/N  or  the  g-times  trimmed  mean  is  '^'^ 

The  g-times  Winsorized  mean  and  sum  of  squared  deviations  are, 
respectively, 

■'^  i=Sr+l 

■-   '    -i       •         .  ■• 

and, 

N-g 
i=g*i 

Tukey  and  McLaughlin  (1963)  found  that  the  Winsorized 
sample  variance  is  a  suitable  estimate  of  the  variance  of  the 
trimmed  mean.  They  suggested  the  ratio  of  the  trimmed  mean  to 
the  Winsorized  variance 

t     =   -  t^o  ::. 

r  ^^^.^^  11//  (14) 

{N-2g)  {N-2g-l)  ..  , 

as  an  alternative  to  the  single  sample  t  test.     In  equation 
(14)        is  the  hypothesized  mean;  the  null  distribution  of 
is  approximated  by  Student's   t  with  N  -  2g  -   1  degrees  of 
freedom. 
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Yuen  and  Dixon  (1973)  suggested  a  trimmed  t  test  using 


y2tg  -  y^tg  - 


SSD.^^^SSD,^^  1        ^         1  '  (15) 

{N^-2g^)  +  {N^-lg^) -2     (N^-2g^  (^2-2^2) 


as  an  alternative  to  the  independent  samples  t  test.  In 
equation  (15)  ju^  -  is  the  hypothesized  mean  difference  and 
numerical  subscripts  differentiate  groups.  The  distribution 
of  t^g  is  approximated  by  Student's  t  with  (N^  -  2g^)  +  (Nj  - 
2g2)    -  2  degrees  of  freedom. 

Yuen  (1974)  suggested  replacing  Welch's  test  on  two  means 

by 


t*  = 


[  ^^^1..^  ,  ^^^2w  11/2  (16) 

{N^-2g^)  {N^-2g^-l)     {N^-2g^)  {N^-2g^-l) 


which  is  distributed  approximately  as  Student's  t  with  degrees 
of  freedom  f  calculated  from 


.  (1-c) 


f       {N^-2g^-l)     {N^-2g^-l)  ' 

where 


(17) 


^  ^   (N^-2g^)  iN^-2g^-l} 


(18) 


{N^-2g^)  {N^-2g^-l)      {N^-2g^)  {N^-2g^-l) 

Yuen  (1975)  compared  the  power  of  the  one-sample 
Winsorized  t,  trimmed  t  and  Student's  t  for  long-tailed 
distributions.  The  trimmed  t  and  Winsorized  t  were  more 
powerful  than  Student's  t  for  sample  sizes  10  or  above,  when 


the   degrees    of   trimniing  was    appropriate.      Yuen   and  Dixon 

(1973)  investigated  the  power  of  the  two-sample  trimmed  t  test 
with  equal  population  variances.  The  power  of  trimmed  t  never 
exceeded  the  power  of  Student's  t  under  normality,  but  with 
sample  sizes  of  greater  than  7  the  loss  of  power  was  small. 
Under  contaminated  normal  distributions  and  sample  sizes  not 
less  than  10,  the  trimmed  t  was  superior  to  Student's  t.  Yuen 

(1974)  reported  that  in  the  case  of  unequal  sample  size  and 
variance  the  trimmed  t  is  superior  to  Welch's  test  for  long- 
tailed  distributions  and  is  comparatively  more  powerful  if  the 
larger  sample  size  goes  with  the  larger  variance  rather  than 
with  the  smaller  variance. 

Under  the  violation  of  the  sphericity  assumption  and/or 
normal  distribution  assumption,  Wilcox  (1993)  compared  Type  I 
error  and  power  rates  of  usual  e-adjusted  F  test  (F)  and  e- 
adjusted  F  test  using  trimmed  means  and  Winsorized  variances 
(D)  in  repeated  measures  designs.     Wilcox  included  K  =  4  and 
5,  n  =  10,  20,  and  40,  and  10  and  20  percent  trimming.     For  K 
=  5  the  correlation  matrices,   p.,^,    included  p-,^    =  p,   j   ^  k, 
with    p    =    0,    0.1,    0.5,    and    0.9,    which    all    have    e    =  1. 
Simulations  were  also  run  for  all  of  the  correlation  matrices 
considered  by  Huynh  and  Feldt   (1976),   which  have  e   =  0.363, 
0.522,  0.752,  0.831,  and  1.     For  K  =  4,  the  same  correlations 
were  used  but  with  the  last  row  and  column  of  the  correlation 
matrix  removed.    Nonnormal  distributions    were  generated  using 
the  g-and-h  distribution  in  Hoaglin  (1985)  and  included  both 
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synunetric  and  asymmetric  long-tailed  distributions.  The 
degrees  of  freedom  for  the  e -adjusted  F  test  using  trimmed 
means  and  Winsorized  variances  were  estimated  to  be  =  e(K  - 
1)  and        =  e(K  -  l)(n  -  2g  -  1). 

Wilcox  (1993)  concluded  that  there  is  close  agreement 
between  D  and  F  for  Type  I  error  rate.  Both  procedures  were 
too  conservative  when  n  is  small  and  t  is  close  to  one.  When 
e  is  relatively  small,  the  F  test  can  have  a  Type  I  error  rate 
slightly  higher  than  the  nominal  level  when  the  distributions 
are  normal.  In  contrast,  D  has  a  Type  I  error  rate  slightly 
below  the  nominal  level.  The  power  of  D  is  substantially 
better  than  F  when  distributions  have  heavy  tails  and  the 
advantage  in  using  D  seems  to  increase  for  asymmetric 
distributions. 

Pairwise  Multiple  Comparison  Procedures 

Rejection  of  the  omnibus  null  hypothesis  Hq   :  jj-^   =  pi^  = 

-  -  '   =  Uj,   ^o^^         ^  1^2  =  "  -   =  Uk  °^  ^02'    i^i*   =  h*   =  =  l^K 

is  often  followed  by  pairwise  multiple  comparisons  of  the  p^s, 
to  determine  which  groups   are  different.      There   have  been 
several  studies  of  Type  I  error  rates  of  multiple  comparison 
procedures  developed  for  use  in  repeated  measures   or  split 
plot  designs. 

Maxwell  (1980)  compared  five  pairwise  multiple  comparison 
procedures  that  can  be  used  in  repeated  measures  designs. 


1.   In  the  Tukey  procedure  the  test  statistic  is 


Q  = 


(19) 


{MS^Jn)  ' 


where  and  y,^,  are  the  sample  means  for  the  kth  and  k'th 
level  of  repeated  measures  and  MSg.g  is  the  error  term  for  the 
repeated  measures  ANOVA.  The  q  value  is  compared  to  the  upper 
a  point  of  the  Studentized  range  distribution  based  on  (n  - 
1)(K  -  1)  degrees  of  freedom.  Tukey 's  procedure  is  based  on 
three  assumptions:  (a)  normality  of  the  population 
distribution  for  each  of  the  J  populations,  (b)  equal 
variances  for  each  of  the  populations,  and  (c)  equal 
population  variances  for  all  differences  between  pairs  of 
means,  that  is,  sphericity  condition.  The  validity  conditions 
are  the  same  as  those  for  the  omnibus  F  test  in  the  repeated 
measures  design. 

2.  In  the  modified  Tukey  procedures  the  test  statistic  is 


In  one  procedure  the  q  value  is  compared  to  a  critical  value 
of  the  Studentized  Range  Statistic  with  {n  -  1){K  -  1);  in  the 
other  the  critical  value  has  n  -  1  degrees  of  freedom. 

3.  In  the  Bonferroni  approach  the  test  statistic  is  t  = 
q/v^2  and  the  critical  t  has  n  -  1  degrees  of  freedom  and  per- 
comparison  level  a      =  2a^„   /  K{K  -  1). 


Q  = 


Vk-Yk' 


(20) 


V'(s/+V-2s^jt')/-" 
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4.  In  the  Simultaneous  Confidence  Interval  (SCI) 
procedure  the  test  statistic  is  t  and  the  critical  value  is 

±1  Fia,K-l,n-K.l)]''\  (21) 

(rj-A+1) 

The  factors  in  Maxwell  (1980)  were  the  number  of  repeated 
measures  (K  =  3,  4,  and  5),  the  sample  size  (n  =  8  and  15), 
and  e.  The  quantity  e  ranged  from  0.49  to  1.00,  the  variation 
in  e  depended  on  K.  Maxwell  concluded  that  the  SCI  approach 
is  too  conservative,  especially  when  n  is  small  relative  to  K 
and  that  the  use  of  the  Tukey  procedure  under  violation  of 
sphericity  leads  to  inflated  Type  I  error  rates.  Mitzel  and 
Games  (1981)  reported  the  same  result  in  a  design  with  n 
equals  20.  Maxwell  (1980)  recommended  the  Bonferroni  approach 
as  the  most  powerful  among  the  procedures  that  control  the 
Type  I  error  rate,  a  =  .05. 

Keselman  and  Keselman  (1988)  investigated  four  procedures 
that  can  be  used  to  make  pairwise  comparisons  of  the  elements 
of  the  vector  of  /i  =  N"''  lnj^^  or  of  ji*  =  J"''  Iji.  in  a  split 
plot  design.  Let  y  =  N"^  En^y^  and  y*  =  J"""  Ey..  Keselman  and 
Keselman  included  the  Tukey  procedure.  To  test  contrasts  on 
ji  the  test  statistic  is 


Q  =     ,  ■  .  ;  .  (22) 


To  test  contrasts  about  fi*  the  test  statistics  is 


In  both  cases  the  critical  value  is  the  upper  a%  point  of  the 
Studentized  range  distribution  with  (W  -  J) (K  -  1)  degrees  of 
freedom.  Keselman  and  Keselman  also  included  a  modified  Tukey 
procedure  in  which  the  statistic  for  contrasts  on  ^  is 

g=     ,  (24)' 

and  the  test  statistic  for  contrasts  on  ^*  is 

Yk-Vk'* 


Q  = 


(25) 


In  equation  (24)  and  equation  (25),  s^^^  and  s,^,^  are  pooled 
variances  and  s,^,^,  is  a  pooled  covariance.  The  critical  value 
is  the  upper  a%  point  of  the  Studentized  range  distribution 
with  N  -  J  degrees  of  freedom.  Keselman  and  Keselman  also 
included  two  procedures  that  use  t  =  q/\^2  as  the  test 
statistic.  In  the  first,  the  Bonferroni  procedure,  the 
critical  value  is  a  t  critical  value  with  N  -  J  degrees  of 
freedom  and  a^^  =  2a^„  /  K{K  -  1).  In  the  second,  the  SCI 
procedure,  the  critical  value  was        r  V '*  .:• 


In  Keselman  and  Keselman  (1988),  J  =  3  in  all  conditions. 
The  population  v-c  were  either  equal  or       =  SE^  and  E3  -  5E^ . 
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There  were  three  other  factors:  (a)  fC  =  4  and  8,  (b)  e  =  .40, 
.75,  and  1.0,  and  (c)  =  3,  11,  16;  7,  10,  13;  10,  10,  10; 
13,  10,  7;  and  16,  11,  3.  Keselman  and  Keselman  concluded 
that  when  heterogenous  v-c  matrices  are  combined  with  unequal 
group  sizes,  none  of  the  four  procedures  controls  the  Type  I 
error  rates  involving  comparisons  of  unweighted  means  (y*). 
But,  when  weighted  means  (y)  are  used,  the  Bonferroni  and  SCI 
procedure  control  the  Type  I  error  rate.  The  Bonferroni  has 
more  power  than  the  SCI  and  the  power  advantage  increases  as 
the  number  of  repeated  measures  levels  increases. 

Keselman,  Keselman,  and  Shaffer  (1991)  investigated  four 
procedures  that  can  be  used  to  make  multiple  pairwise 
comparisons  of  the  elements  of  the  vector  /i*  =  J"''  E^.  in  a 
split  plot  design.  Keselman,  Keselman,  and  Shaffer  included 
the  Studentized  maximum  modulus  critical  value,  ^  where 
M  is  the  upper  100(1  -  a)  percentile  with  parameters  c  =  K(K  - 
l)/2  and  Satterthwaite  (1941)  approximate  degrees  of  freedom 
V .  Estimated  Satterthwaite  approximate  degrees  of  freedom  are 
given  by 


V  = 


[(s,,^.s,,^^-2s,,,A/n,]  -  •  (27) 


They  also  included  the  Bonferroni  critical  value,   t  . 

'       a/ (2c),       V ' 

where  t  is  the  upper  100 [1  -  a/ (2c)]  percentile  of  Student's 
t  distribution  with  degrees  of  freedom  v  given  by  equation 
(27),   and  the  Studentized  range  critical  value,   q^  ^ 
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with  degrees  of  freedom  v  obtained  from  equation  (27). 
Finally,  they  included  an  analog  of  Cochran's  (1964)  critical 
value, 

^<tK=—  •  (28) 

5^  [  (S^.j,2  +  S^.j,/2-2s^.j,^.)/i3j.] 

j 

The  test  statistic  in  all  cases  was 


52  Wjk-Vjk') 


(29) 


where  Sj|^  denotes  sample  variance  for  group  j  and  k  occasion. 

In  Keselman,  Keselman,  and  Shaffer  (1991)  J  =  3  in  all 
conditions.  Population  v-c  matrices  were  equal,  or  =  1 .  5E-, 
and  =  2E^,  or  =  3E^  and  E3  =  5E^ .  Two  population  shapes, 
normal  and  chi-square  with  three  degrees  of  freedom,  were 
included.  There  were  three  other  factors:  (a)  K  =  4  and  8, 
(b)  e  =  .40,  .57,  .75,  and  1.0,  and  (c)  =  6,  10,  14;  8,  10, 
12;  10,  10,  10;  12,  10,  8;  14,  10,  6  for  N  =  30,  and  -  9, 
15,  21;  12,  15,  18;  15,  15,  15;  18,  15,  12;  21,  15,  9  for  N  = 
45.  Keselman,  Keselman,  and  Shaffer  concluded  that  the 
statistic  in  equation  (29)  with  a  Studentized  range,  a 
Studentized  maximum  modulus,  or  a  Bonferroni  critical  value, 
based  on  Satterthwaite ' s  degrees  of  freedom,  provides 
reasonable  control  over  the  Type  I  error  rates.  Use  of  the 
Bonferroni  critical  value  resulted  in  the  smallest  Type  I 
error  rate  and  the  Studentized  maximum  modulus  critical  value 
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provided  a  more  powerful  test  than  did  the  Bonferroni  critical 
value. 

Keselman,  Keselman,  and  Shaffer  (1991)  found  that  the 
Type  I  error  rate  decreased  as  e  decreased  for  both  normal  and 
nonnormal  data  regardless  of  the  degree  of  heteroscedasticity . 
When  the  Studentized  range  critical  value  was  used,  Type  I 
error  rates  were  larger  for  the  nonnormal  conditions.  The 
tests  using  the  Studentized  maximum  modulus  or  the  Bonferroni 
critical  values  were  not  consistently  affected  by 
ndnnormality .  With  the  Studentized  range  critical  value  Type 
I  error  rates  increased  as  the  number  of  repeated  measures 
levels  increased.  This  was  not  the  case  for  the  tests  using 
the  Studentized  maximum  modulus  or  the  Bonferroni  critical 
value.  ^  . 

Keselman  (1994)  investigated  28  pairwise  multiple 
comparison  procedures: 

1.  Four  adaptations  of  Hayter's  (1986)  modification  of 
Fisher's  LSD  test  were  used.  Four  tests  of  the  within- 
subjects  main  effect  were  employed:  e-adjusted,  e-adjusted,  e- 
ad justed  and  Hotelling's  T^.  Pairwise  comparisons  were 
conducted  using  equation  (29)  and  (q^  v)/^2. 

2.  An  adaptation  of  Hochberg's  (1988)  step-up  procedure 
using  equation  (29). 

3.  An  adaptation  of  Shaffer's  (1986)  step-down  procedure 
using  equation  (29). 
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4.  Four  adaptations  of  Shaffer's  sequentially  rejective 
procedure.    Four  tests  of  the  within-subjects  main  effect  were 

^  —   

employed:  e-adjusted,  e-adjusted,  e-adjusted,  and  Hotelling ' s 
T^.     Pairwise  comparisons  were  conducted  using  equation  (29). 

5.  Peritz ' s  (1970)  procedure  using  equation  (29)  as  a 
range  statistic  and  q/V2  as  critical  values,  where  q  denotes 
a  Studentized  range  critical  value.  .    •  v."^-^ 

6.  Three  adaptations  of  Peritz ' s  (1970)  test  using  e- 
adjusted,  e-adjusted,  e-adjusted  tests  and  equation  (29)  for 
pairwise  comparisons. 

7.  Multiple  F  tests  using  e-adjusted,    e-adjusted,  e- 
adjusted  tests,  with  the  error  mean  square  computed  by  using 
the    data    for    the    treatment    levels    being    compared.  For 
pairwise  comparison  equation  (29)  was  used. 

8.  Multiple  tests.  For  pairwise  comparison  equation 
(29)  was  used. 

9.  An  adaptation  of  the  Ryan-Welsch  test  (Welsch,  1977) 
using  equation  (29)  and  q/\/'2. 

10.  Four  adaptations  of  Shaffer's    (1979)   test.  Four 
tests  of  the  within-subjects  main  effect  were  employed:  e- 
adjusted,      c-adjusted,      e-adjusted,      and     Hotelling' s  T^. 
Pairwise   comparisons   were   employed  using  equation    (29)  and 
q//2. 

11.  An  adaptation  of  Welsch 's  (1977)  step-up  test. 
Pairwise  comparisons  were  employed  using  equation  (29)  and 
q//2.      Because  v  depends  on  the  particular  comparison,  the 
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Welsch  critical  values  were  forced  to  follow  a  monotone 
sequence.  That  is,  if  the  critical  value  for  p  means  was 
smaller  than  the  critical  value  for  p  -  1  means,  the  former 
critical  value  was  set  equal  to  the  latter. 

12.  An  adaptation  of  Tukey's  test  using  equation  (29) 
a'^d  q^  |<^  ^.  ,    ■  . 

13.  A    Studentized    maximum    modulus    procedure  using 
equation  (29)  and         k(k-1)/2,  v 

14.  A   Bonferroni   procedure    using   equation    (29)  and 

^o/K(K-1),  V* 

In  all  conditions  there  were  J  =  3  groups.  Let  E  denote 
the  dispersion  matrix  for  the  group  in  which  the  elements  of 
the  dispersion  matrix  were  the  smallest.  The  dispersion 
matrix  was  3E  for  one  of  the  other  two  groups  and  was  5E  for 
the  remaining  group.  Keselman  manipulated  eight  factors. 
Four  of  the  factors  were:  (a)  K  =  4  or  8,  (b)  e  =  .40  and  .75, 
(c)  N  =  30  and  45,  (d)  =  10,  10,  and  10;  8,  10,  12;  and  6, 
10,  14  for  N  =  30,  and  15,  15,  and  15;  12,  15,  and  18;  and  9, 
15,  and  21  for  N  =  45.  Keselman  included  positive  and 
negative  pairings  of  unequal  sample  sizes  and  unequal 
dispersion  matrices  and  multivariate  normal  and  multivariate 
nonnormal      data.  In      the      nonnormal      case  canonical 

transformation  of  the  nonnormal  data  would  result  in  K 
uncorrelated   variables    each   with   a  distribution.  Two 

configurations  of  means  were  included:  ju-i  =  /Jj      ^^3  =  ^^4      '  *  ' 
Pk'  ^1   =  ^2  =  ■  '  ■   =  i^'K-     Keselman  concluded  that 
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the  Welsch  step-up  test  was  robust  to  nonnormality  and  tended 
to  have  better  power  than  the  other  multiple  comparison 
procedures . 

Dunnett  (1982)  investigated  several  pairwise  multiple 
comparison  procedures  that  can  be  used  to  compare  means  from 
J  independent  samples.  Dunnett  considered  intervals  of  the 
form 

V  .-V  i±A*  .  S j       Sji  .  • 

where      ,  denote  estimates  of  location  and  variance  for 

the  jth  sample,  n^  represents  effective  sample  size  for  the 
estimator,   and  represents  a  constant  which  is  to  be 

chosen  so  that  the  desired  joint  confidence  coefficient  1-a  is 
achieved. 

Along  with  the  ordinary  sample  mean  and  sample  variance, 
five  robust  estimators  of  location  and  the  corresponding 
variance  estimate  were  examined.  The  robust  estimators  were 
wave  estimate,  bisquare  estimate,  Hampel  estimate,  10%  trimmed 
mean  with  Winsorized  standard  deviation,  and  Tiku ' s  (1967) 
modified  maximum  likelihood  estimates  using  10%  trimmed 
samples.  In  addition  a  multiple  comparison  extension  of  the 
Wilcoxon  rank  sum  method  was  included. 

The  experiment  was  conducted  for  eight  symmetric 
distributions:  uniform,  normal  (0,1),  t  distributions  with  10, 
4,    or   2   degrees   of   freedom,    Cauchy   distribution,    and  two 


normal  outlier  distributions.  These  were  .9[N(0,1)]  + 
.1[N(0,3)]  and  .9[N(0,1)]  +  .1[N(0,10)].  The  experiment  had 
three  combinations  of  K  and  n:  K  =  4,  n  =  10  or  20  and  K  =  8 
and  n  =  10 .  ^ 

Dunnett  (1982)  concluded  that  the  procedure  using  the 
ordinary  sample  mean  and  sample  variance  becomes  conservative 
with  data  from  long-tailed  distributions  and  it  should  not  be 
used  in  more  than  moderate  departures  from  normality.  The 
procedures  using  trimmed  mean  and  modified  maximum  likelihood 
estimate  did  not  become  conservative.  The  nonparametric 
method  should  not  be  used  if  variance  heterogeneity  is 
excessive.  The  procedure  using  the  wave  estimate,  bisquare 
estimate,  or  Hampel  estimate  should  not  be  used  with  data  from 
short-tailed  distributions.  ;  ,,•» 

Summary  and  Implications  \' 

Based  on  research  by  Algina  and  Oshima  (1994),  Algina  and 
Oshima  (in  press),  Huynh  (1978),  Keselman  and  Keselman  (1988), 
Keselman,  Keselman,  and  Shaffer  (1991),  and  Keselman  (1994) 
the  following  conclusions  can  be  set  forth. 

1.  Under  normality,  the  e -adjusted  test  controls  the  Type 
I  error  rate  for  the  within-subjects  main  effect  on  weighted 
means  even  when  multisample  sphericity  is  violated.  There 
does  not  appear  to  be  any  advantage  to  using  the  GA  or  IGA 
tests.     Although  the  available  results  are  limited,  the 
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conclusion  appears  to  be  true  when  the  data  come  from  a 
multivariate  lognorraal  distribution. 

2.  When  the  unweighted  means  are  used  the  e -adjusted  test 
does  not  exhibit  good  control  of  the  Type  I  error  rate  of  the 
within-subjects  main  effect  for  unequal  samples  sizes  and 
heterogeneous  v-c  matrices.  However,  when  sample  sizes  are 
equal  the  test  provides  good  control  of  the  Type  I  error  rate 
even  when  the  v-c  matrices  are  heterogeneous.  When  the  sample 
sizes  are  not  equal  the  IGA  and  CIGA  tests  control  the  Type  I 
error  rate  much  better  than  does  the  e -adjusted  test. 

3.  For  the  multiple  comparison  tests  with  the  weighted 
means,  Type  I  error  rates  of  Tukey  and  modified  Tukey 
(Maxwell,  1980)  pairwise  multiple  comparison  procedures  are 
affected  under  the  violated  multisample  sphericity,  but  the 
Bonferroni  and  SCI  procedures  have  a  good  control  of  the  Type 
I  error  rate.  The  Bonferroni  procedure  has  more  power  than 
the  SCI  procedure  does  and  the  power  advantage  increases  as 
the  number  of  repeated  measures  levels  increases. 

4.  When  heterogeneous  v-c  matrices  are  combined  with 
unequal  group  sizes,  none  of  the  Tukey,  modified  Tukey 
(Maxwell,  1980),  Bonferroni,  or  SCI  has  a  good  control  over 
the  Type  I  error  rates  involving  comparisons  of  unweighted 
means.  Type  I  error  rate  decreases  as  e  decreases  for  both 
normal  and  nonnormal  data  regardless  of  the  degree  of 
heteroscedasticity.  There  is  a  variety  of  multiple  comparison 
procedures  that  control  the  Type  I  error  rate  for  comparisons 
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of  unweighted  means.  Keselman  (1994)  reports  that  an 
adaptation  of  the  Welsch  (1977)  step-up  test  tends  to  have  the 
best  power. 

Studies  by  Dunnett  (1980),  Tukey  and  McLaughlin  (1963), 
Wilcox  (1992,  1993),  Yuen  (1974),  and  Yuen  and  Dixon  (1973) 
indicate  that  normal  theory  tests  become  conservative  and  have 
reduced  power  with  data  from  long-tailed  distributions. 
Replacing  means  by  trimmed  means  and  variance  by  Winsorized 
variances  tends  to  solve  these  problems. 

The  purpose  of  the  current  study  is  to  investigate  Type 
I  error  rates  and  power  of  multiple  comparison  procedures  that 
can  be  used  in  split  plot  designs.  Both  normal-theory 
procedures  and  alternatives  constructed  by  replacing  means  by 
trimmed  means  and  variances  by  Winsorized  variances  are 
considered. 


CHAPTER  3 
METHODOLOGY 


In  this  chapter,  the  multiple  comparison  procedures  and 
their  test  statistics,  the  design,  and  the  simulation 
procedure  are  described.  The  design  is  based  upon  a  review  of 
relevant  literature  and  upon  the  consideration  that  the 
experimental  conditions  used  in  the  simulation  should  be 
similar  to  those  found  in  educational  research. 

Multiple  Comparison  Procedures 

Three  types  of  multiple  comparison  procedures  were 
employed  in  the  study:  (a)  the  Bonferroni  procedure,  (b) 
Shaffer's  modified  sequentially  rejective  procedure,  and  (c) 
Welsch's  step-up  procedure.  The  implementation  of  the 
procedures  is  described  in  the  next  several  paragraphs. 
Bonferroni  Procedures  • ' ' 

Three    variations    of    the    Bonferroni    procedure  were 
investigated.     In  the  first,  the  test  statistic  was 

In  equation  (31)  y^,^  and  s^^^  are  the  mean  and  variance  of  y,^ 
at  the  jth  level  of  the  between-subjects  factor;  s^,^,^,     is  the 
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covariance  between  y,^  and  y,^,  at  the  j'th  level  of  the  between- 
subjects  factor.  The  critical  value  was  ±t„  /,  „  ,  where  = 
2af„    /  [K(K  -  1)]  and 

Subsequently,  this  procedure  is  referred  to  as  the  Bonferroni 
(B)  procedure. 

In  the  second  procedure  y^i^  and  yj,^,  were  replaced  by  g- 
times  triimned  means  and  s^^,^,  s^^^^.,  and  Sj|^|^,  were  replaced  by 
Winsorized  variances  and  covariances,  respectively.  The 
Winsorized  covariance  is  defined  as 


~^  (nj-2g-l)  '  ^^^^ 


where  Y*^j\,  is  a  Winsorized  data  point  and  y*-,^,  y*.,^,  are 
Winsorized   means.       The    critical    value   was  ±t  .  The 

degrees  of  freedom       are  obtained  by  replacing  s^j,^    and  s^,^,^, 
by  Winsorized  variances   and  covariances,    respectively;  the 
quantities        -   1  were  replaced  by  n^    -  2g  -1.      The  second 
procedure  is  referred  to  as  the  raw  score  trimmed  Bonferroni 
procedure  (Bp^.). 

Let  d,^,^,    =  y^   -  y^.  •       In  the  third  procedure  the  test 
statistic  was 


-1==='  (34) 

where  d*^^^^^.  is  the  trimmed  mean  of  d-^^^..  =  y-^  -  y.,^,  and  s*-^^, 
is  the  Winsorized  standard  deviation  of  d^^^^^.  =  y-|^  -  y.,^, . 
The  critical  value  was  ±t„    „  where 

^  nj-2g-l 

The  third  procedure  is  called  the  trimmed  difference  score 
Bonferroni  procedure  (Bp-^).  "  i'  .  j      ,    ••  '\  • 

The  Bonferroni  procedures  were  included  primarily  because 
multiple  comparison  procedures  using  the  Bonferroni  critical 
value  are  well  known  and  commonly  employed. 
Modified  Sequentially  Rejective  Bonferroni  procedures 

Six  variations  on  Shaffer's  modified  sequentially 
rejective  Bonferroni  (MSRB)  procedure  were  included  in  the 
study.  The  steps  in  the  MSRB  procedure  will  be  described  in 
the  context  of  the  first  variation  included  in  the  study. 
Then  the  other  MSRB  procedures  will  be  described. 

The  MSRB  procedure  is  a  modification  of  the  Bonferroni 
procedure  which  is  implemented  in  several  steps.  The  first 
step   is   to  test   Hqj    :  =   •••    =  p^^* .       in  the   first  MSRB 

procedure,  was  tested  by  using  the  e-adjusted  test.  If  Hqj 
is    not    rejected,     all    pairwise    comparisons    are  declared 
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nonsignificant.  If  Hgj  is  rejected,  test  statistics  for  the 
K{K  -  l)/2  pairwise  comparisons  are  calculated  and  ranked  by 
their  p  values  from  smallest  to  largest.  In  the  first  MSRB 
procedure  the  test  statistic  was  calculated  by  using  equation 
(31)  and  its  degrees  of  freedom  were  calculated  by  using 
equation  (32).  The  critical  value  for  the  test  statistic  with 
rank  i  (i  =  1,  •  •  K{K  -  l)/2)  is  ±t„^y2c„v,-  ^  this 
study,  K  =  4;  for  K  =  4,  K{K  -  l)/2  =  6  and  there  are  six  c^ : 
C-,  =  Cj  =  Cj  =  C4  =  3;  C5  =2;  c^  =  1  •  Testing  is  conducted  in 
steps.  If  the  test  statistic  in  the  ith  step  is  not 
significant,  all  subsequent  steps  are  declared  non- 
significant.    The  first  MSRB  procedure  is  denoted  by  S(e). 

The  potential  advantage  of  S(e)  over  B  is  that  the 
critical  values  for  the  S(e)  procedure  are  uniformly  less 
extreme  than  are  the  critical  values  for  B.  The  potential 
disadvantage  is  that  a  lower  ranked  test  statistics  can  be 
declared  nonsignificant  because  of  a  nonsignificant  test  in  a 
prior  step.  Nevertheless,  MSRB  procedures  tend  to  be  more 
powerful  than  Bonferroni  procedures  (Keselman,  1994). 

In  the  second  MSRB  procedure,  Hqj  is  tested  by  using  a 
trimmed  e-adjusted  test.  That  is,  in  the  calculation  of  F  and 
e,  trimmed  means  replace  means  and  Winsorized  variances  and 
covariances  replace  variances  and  covariances.  In  subsequent 
steps  the  test  statistics  and  degrees  of  freedom  employed  in 
B^j  are  used.  The  second  MSRB  procedure  is  denoted  by  S(e)pT- 
The  third  MSRB  procedures,  denoted  by  S(t)Qj,    uses  the  trimmed 


e-adjusted  test  in  the  first  step;  in  subsequent  steps  the 
test  statistics  and  degrees  of  freedom  used  in  Bp^   are  used. 

The  fourth  MSRB  procedure  is  the  same  as  the  first, 
except  the  CIGA  test  developed  by  Algina  and  Oshima  (in  press) 
replaces  the  e-adjusted  test.  The  fourth  MSRB  test  is  denoted 
by  S(C).  In  the  CIGA  test  the  statistic  for  testing  H^*  is 
the  usual  F  statistic  given  by  equation  (5).  The  critical 
value  is    iJp^^^^  where  b,  h',  and  h  are  described  as  follows. 

Let  R  denote  a  [ {K  -  1)  x  K]  matrix  with  rows  that  are 
orthonormal    contrast   vectors,  the    (K   x    1)    sample  mean 

vector  for  group  j,  the  (K  x  K)  sample  dispersion  matrix 
for  group  j.  In  order  to  adjust  critical  value  for  violations 
of  raultisaraple  sphericity,  the  GA  test  uses  Sf^jj  ^  as  the 
critical  value.  The  arguments  of  the  critical  value  are 
defined  in  terms  of  the  Sg  matrices.  Define 

Based  on  Box  (1954)  and  using  the  results  in  Huynh  (1978) 
Algina  and  Oshima  (in  press)  showed  that  for  the  test  of  H 
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:  p,*   =  •  •  •  = 


(N-J)  triRSR') 


(nj-l)  triRSjR')  '  (37) 
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and 


£  =  — i  .  (39) 

j 

Huynh  proposed  the  IGA  test  which  replaces  h  and  h'  by 
statistics   that  are   intended  to  reduce  the  underestimation 
which  occurs  when  multisample  sphericity  nearly  holds.  Let 
=   tr(RSjR')    and         =    tr(RSjR' )^        In   the    IGA   test,    h  is 

—  A,  /\ 

replaced  by  h  =  r]/6  where 


and 


Based  on  the  expression  for  e  (Huynh  and  Feldt,  1976)  Huynh 
suggested  replacing  h '  by  "  «  ?  %  .  '  ^ 

f^-  %    V  (42) 

Lecoutre  (1991)  presented  a  correction  to  the  IGA  test  by 
replacing  N  in  the  numerator  of  h'  by  N  -  j  +  l.     it  is  called 
the  CIGA  test. 

In  the  fifth  MSRB  test,   denoted  by  8(0)^^,    a  raw  score 
trimmed  CIGA  test   is   used  to   test   Hqj.       In   this   test,  the 


means,  variances  and  covariances  used  in  the  CIGA  test  were 
replaced  by  trimmed  means  and  Winsorized  variances  and 
covariances,  respectively.  The  test  statistics  and  degrees  of 
freedom  employed  in  the  subsequent  steps  are  the  same  as  in 
8(6)^^.  In  the  sixth  MSRB  procedure,  denoted  by  S(C)p^,  the 
trimmed  CIGA  test  is  again  used  in  the  first  step.  The  test 
statistics  and  degrees  of  freedom  employed  in  the  subsequent 
steps  are  the  same  as  in  S(e)p^. 
Welsch's  Step-up  Procedures 

Two  variations  of  Welsch's  step-up  procedure  were 
included  in  the  study.  Steps  in  Welsch's  step-up  procedure 
will  be  described  in  the  context  of  the  first  variation 
included  in  the  study.  Then  the  other  procedures  will  be 
described. 

In  Welsch's  step-up  procedure  the  means  for  all  K  levels 
of  the  within-subjects  factor  are  ordered  from  smallest  to 
largest.     The  procedure  begins  by  testing  all  2-ranges, 
(^*(k+i)     "  ^*(k))     (1  ^  k  <  K-1) ,  of  the  ordered  treatment  means 
of  each  level  of  the  within-subjects  factor,  Y*^^)    <  Y*(2)    <  • 
■   ■  -  ^*(K)-  the  2-range  is  significant,  the  members  of  the 

corresponding  pair  of  treatments  are  declared  different.  Also 
declare  the  p-ranges  containing  that  2-range  significant  and 
all  sets  of  treatments  which  contain  that  significant  subset 
as  heterogeneous  by  implication  without  further  tests.  If  at 
least  one  2-range  is  not  significant,  then  proceed  to  test  3- 
ranges.     In  general,  test  a  p-range  {Y\^^^_^^      -  Y*(^j)     (1  <  k 
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<  K  -  p  +  1,  2  <  p  <  K) ,  if  that  p-range  is  not  declared 
significant  by  implication  at  earlier  steps.  If  Y*(p+|^_^) 
Y*(|^)  is  significant,  then  declare  the  pair  of  treatments 
corresponding  to  Y*(p^.,^_^)  and  Y*(|j)  as  different.  Also  declare 
all  sets  of  treatments  containing  that  subset  of  p  treatments 
as  heterogeneous  and  the  corresponding  g-ranges  containing 
that  p-range  for  all  g  >  p  as  significant  by  implication 
without  further  tests.  Welsch  proposed  using  -  pa/K  for  2 
p  <  K  -  2,  and  a,^.^  =  =  a.  Welsch  (  1977b)  tabulated  the 
critical  values  ^^^.v  (2  ^  P  ^  i^)  based  on  Studentized  range 
statistics  for  k  =  2 ,  3 ,  •   •   •  ,  10 ,  v  =  5 ,  6 ,  •  •  •  ,  20 ,  24,  30, 

40,    60,    120,   t»,    and  a  =    .05  which  he  obtained  using  Monte 

„         ■  ■  •- 

Carlo  simulations. 

In  the  first  variation  of  Welsch 's  step-up  procedure, 
denoted  by  W,  the  test  statistic  was  calculated  using  equation 
(31).       The    critical    value    is  In    the  second 

variation  of  Welsch 's  procedure  the  ordered  means  used  in  the 
first  variation  are  replaced  by  Ytjic/"^  where  y^-^.  is  the 
trimmed  mean  for  the  jkth  cell  of  the  design.  The  test 
statistic  for  comparing  means  is  the  test  statistic  used  in 

Bp,..        The    critical    value    is  \^^_^J>j2  .      This    procedure  is 
referred  to  as  the  raw  score  trimmed  Welsch  procedure  (W^^) . 
In  both  procedures  the  critical  values  were  forced  to  follow 
a  monotone  sequence.      That  is  \„   „    was  set  equal  to  £ 

Whenever  <  ( 3  <  p  <  X  and  s  =  1 ,   2,  3). 


The  potential  advantage  of  Welsch's  step-up  procedures  is 
better  power  than  Shaffer's  procedures  because  the  step-up 
procedures  declare  significance  by  implication;  in  contrast 
Shaffer's  procedures  declare  nonsignif icance  by  implication. 

Design 

Six  factors  were  manipulated  to  investigate  the 
performance  of  the  various  multiple  comparison  procedures. 
These  factors  were  (a)  distribution  of  the  generated  data,  (b) 
sphericity  of  the  common  v-c  matrix,  (c)  degree  of  the 
heteroscedasticity  of  the  v-c  matrices,  (d)  total  sample  size, 
(e)  degree  of  the  sample  size  ratio,  and  (f)  the  configuration 
of  the  nonnull  means.  There  were  J  =  2  levels  for  the 
between-subjects  factor  and  J=C  =  4  levels  for  the  within- 
subjects  factor  in  all  conditions.  .  ■ 

Distribution  of  the  Generated  Data  ' 

In  one-third  of  the  conditions  the  data  in  each  group 
were  sampled  from  multivariate  normal  distribution.  The  g- 
and-h  method  (Hoaglin,  1985)  was  used  to  simulate  nonnormal 
data.  Micceri  (1989)  reported  distributional  characteristics 
for  440  data  sets  from  education  and  psychological  research. 
He  reported  that  32.5%  had  at  least  one  heavier  tail  than  the 
extreme  contamination  and  16.6%  had  heavier  tail  than  double 
exponential  distribution  (kurtosis  =  6.0).  The  extreme 
contamination  represents  more  than  100  times  the  expected 
observations  of  normal  distribution  over  3  standard  deviations 
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from  the  mean.  Based  on  Micceri,  in  one-third  of  the 
conditions  the  data  in  each  group  were  sampled  from  the  g-and- 
h  distribution  with  g  =  0  and  h  =  .109  approximating  double 
exponential  distribution.  The  results  in  Wilcox  (1993) 
indicate  that  methods  based  on  trimmed  means  are  more  powerful 
than  those  based  on  untrimmed  means  when  the  data  are  very 
long  tailed.  Based  on  Wilcox's  results,  in  one-third  of  the 
conditions  the  data  in  each  group  were  drawn  from  g-and-h 
distribution  with  g  =  0  and  h  =  .35. 
Sphericity  of  the  Common  V-C  Matrix  (e)  .  .  ■ 

Box's  (1954)  correction  factor,  e,  was  used  to  guantify 
the  degree  of  departure  from  the  sphericity  assumption.  Two 
levels  of  e  were  included  to  investigate  the  effects  of 
nonsphericity .  These  were  e  =  .96,  and  e  =  .75.  Dispersion 
matrices  were  taken  from  Keselman  and  Keselman  (1990).  Table 
1  contains  the  dispersion  matrices  for  each  value  of  £ .  To 
achieve  comparability  across  the  simulation  conditions, 
dispersion  matrices  with  average  diagonal  value  of  10  and 
average  off -diagonal  value  of  5  were  used. 
Degree  of  Heteroscedasticity  of  the  V-C  Matrices 

Conditions  in  which  the  v-c  matrices  were  egual  in  both 
groups  and  conditions  in  which  the  v-c  matrices  were  not  egual 
were  included.     In  the  former  conditions,  the  v-c  matrix  for 
both  groups  was  one  of  the  matrices  in  Table  1.    Two  levels  of 
dispersion  matrix  ineguality  were  included.     In  the  first,  T,^ 
=  (2/3)j:  and       =  (4/3)1]  where  E  is  one  of  the  matrices  in 
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Table  1 

V-C  Matrices  for  e  =  .96  and  4 
/75 


c  v-c  matrix 

0.96  12.0     6.0     5.0  5.0 

10.0     5.0  4.0 
10.0  5.0 
8.0 

0.75  18.0     8.0     6.0  4.0 

8.0     5.0  4.0 
7.0  3.0 
7.0 


Table  1.     In  the  second,        =  (1/3)E  and        =  (5/3)E.  These 
levels  of  dispersion  are  referred  to  as  1:1,   1:2  and  1:5  in 
subsequent  tables. 
Total  Sample  Size  (N) 

Two    levels    of    total    sample    sizes    were  investigated. 
These  were  N  =  40  and  N  =  60. 
Sample  Size  Ratio  ( n., ,  in^X 

Conditions  with  equal  sample  sizes  for  each  between- 
subjects  level  and  conditions  with  unequal  sample  sizes  were 
investigated.  Both  moderate  and  large  degrees  of  sample  size 
inequality  were  included.  For  W  =  40  the  sample  size 
combinations  were  n^,  n2  =  28,  12;  20,  20;  and  12,  28.  For  N 
=  60  the  sample  size  combinations  were  n^,  nj  =  42,  18;  30, 
30;  and  18,  42.  Thus,  there  were  three  levels  of  the  sample 
size  combinations  factor  for  each  total  sample  size  N  =  40  and 
N  =  60,  which  are  referred  to  as  levels  A  to  C  in  subsequent 
tables.  For  those  conditions  involving  unequal  between- 
subjects  group  sizes  and  unequal  v-c  matrices,  both  positive 
and  negative  pairings  of  between-subjects  sample  sizes  and  v-c 
matrices  were  included.  A  positive  pairing  refers  to  the  case 
in  which  the  group  with  larger  sample  size,  nj,  is  associated 
with  the  v-c  matrix  containing  the  larger  element  values;  a 
negative  pairing  refers  to  the  case  in  which  the  larger  sample 
size,  nj,  is  associated  with  the  v-c  matrix  with  the  smaller 
element  values. 


51 

The  Configuration  of  the  Nonnull  Means 

To  investigate  power,  three  configurations  of  nonnull 
means  were  examined  (Ramsey,  1978):  (a)  a  minimum  range 
configuration,  which  occurs  when  half  of  the  means  are  equal 
and  the  remaining  half  are  also  equal  but  different  from  the 
first  half;  (b)  a  maximum  range  configuration,  which  occurs 
when  there  is  one  mean  at  each  extreme  of  the  range  and  the 
remaining  means  equal  the  average  of  these  two  extreme  values; 
and  (c)  an  equally  spaced  range  configuration,  which  occurs 
when  the  means  are  equally  spaced  across  the  range. 

When    E    satisfies    C'EC    =  with    C'C    =    1,^.^ ,  the 

noncentrality  parameter  in  the  split  plot  design  with  two 
levels  in  between-subjects  factor  can  be  shown  to  be      ^     "  " 

nn   t  (^^."-^^^)      '  " 

where  p*  =  (^^|^  +  ^2k)/2/  ^ik  is  the  mean  of  the  ith  level  of 
the  between-subjects  factor  and  kth  level  of  the  within- 
subject  factor,  n^  is  the  number  of  sample  for  the  ith  level 
of  the  between-subjects  factor,  and  jj  is  the  mean  of  ^,^*s. 
For  each  sample  size  combination,  the  value  of  the 
noncentrality  parameter  necessary  to  produce  omnibus  power  of 
.50  for  the  case  in  which  a  =  0.05  and  v-c  matrices  are  equal 
was  calculated.  Then,  the  means  for  each  level  of  the  within- 
subjects  factor  were  determined  by  using 


V   (Jij,*2_-jj2)     =    iVj^;^_52_  ;^v.v^  (44) 

The  ^|^*s  are  assumed  to  sum  to  zero  and  monotonically  increase 
from  k  =  1  to  K.  The  resulting  means  are  presented  in  Table 
2. 

Simulation  Procedure 

Data  Generation 

The  data  for  each  condition  that  involved  multivariate 
normal  data  were  generated  using  the  following  steps: 

1.  For  the  j'th  level  of  the  between-subjects  factor  Zj, 
an  n-  X  4  matrix  of  independent  normally  distributed  variates 
was    generated.       The    NORMAL    function    in    SAS    was    used  to 
generate  all  variates.  ■  ^ 

2.  The  matrix  Zj  was  transformed  to 

Xj  =  ^  +  d.ZjU*  ,  .  (45) 

where  U  is  a  lower  triangular  matrix  satisfying  the  equality 
=  UU',  dj  is  a  constant  selected  to  simulate  the  required 
degree  of  heteroscedasticity,  and  ^  is  an  n-  x  4  matrix  of 
means  selected  to  simulate  the  required  configuration  of 
means . 

The  data  from  nonnorraal  distributions  were  generated 
using  the  g-and-h  distribution  suggested  by  Tukey  (1977)  and 
developed  by  Hoaglin  (1985).  This  family  of  distributions  is 
attractive  in  simulation  studies  because  those  distribution 
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Table  2 

Means  of  Repeated  Measures  for  each  Sample  Size  Arrangement 
(S)    to    get    50    percent    Power    in    Omnibus    F    Test    when  all 
Assumptions  for  omnibus  F  Test  are  satisfied 


configuration 
of  the  nonnull 


means 

S 

^1 

^2 

^3 

minimum  range 

a,  c 

-0 

.470918 

-0. 

470918 

0. 

470918 

0 

.470918 

means 

b 

-0 

.431603 

-0. 

431603 

0. 

431603 

0 

.431603 

d,f 

-0 

.382238 

-0. 

382238 

0. 

382238 

0 

.382238 

e 

-0 

.350327 

-0. 

350327 

0. 

350327 

0 

.350327 

maximum  range 

a,  c 

-0 

.665978 

0 

0 

0 

.665978 

means 

b 

-0 

.610379 

0 

Q 

0 

.610379  .. 

-0 

.540567 

0 

.,  0 

0 

.540567 

e 

-0 

.495438 

0 

0 

0 

.495438 

equal  space 

a,  c 

-0 

.631802 

-0. 

210601 

0. 

210601 

0 

.631802 

means 

b 

-0 

.579056 

-0. 

193019 

0. 

193019 

0 

.579056 

d,f 

-0 

.512827 

-0. 

170942 

0. 

170942 

0 

.512827 

e 

-0, 

.470013 

-0. 

156671 

0. 

156671 

0, 

.470013 

Note.  a:n.  =  12  and  28;  b:n-  =  20  and  20;  c:n.  =  28  and  12; 
d:n^  =  18  and  42;  e:n.   =  30  and  30;   f:n^   =  42  and  18. 


54 

shapes  are  determined  by  a  small  number  of  parameters,  a  wide 
spectrum  of  distributions  can  be  approximated,  and  simulated 
observations    can   be   easily   generated   from   independent  and 
identically  distributed  normal  deviates. 

The    g-and-h   distributions    are    generated   by    a  single 
transformation  of  the  standard  normal  variable  Z^j, 

;r„  =  ^^^i^«lllexp(;=V/2).  (46) 

where  Z^j  is  a  standard  normal  variable,  g  is  a  real  constant, 
and  h  is  a  nonnegative  real  constant.  The  g-and-h 
distributions  are  defined  in  terms  of  their  quantile  function 
relative  to  the  standard  normal  distribution  so  that  if  Z^j  in 
equation  (46)  has  the  value  of  the  pth  quantile  of  the 
standard  normal  distribution,  then  X^^  is  the  pth  quantile  of 
a  g-and-h  distribution  with  the  specified  values  of  g  and  h. 

Martinez  and  Iglewicz  (1984)  and  MacGillivray  (1992) 
investigated  some  properties  of  this  family.  The  g-and-h 
distribution  for  the  subfamily  with  g  =  0  is  ■       ? "  ^  V 


X^j  =  Z^jexp{hZ^^/2)  ,  (47) 

because 


exp(gZ..)-l  "^'^ 
 ^         =  Z,.  +  gZ,//2\  +  g%//3!  +  •  •  (48) 

For  the  case  g  =  h  =  0,  it  follows  easily  that  X^^  =  Z-.,  The 
case  g  =  0  corresponds  to  a  symmetric  distribution,  and 
increased  g  results  in  greater  skewness  to  the  right  for  g  > 
0;,  the  same  amount  of  skewness  to  the  left  can  be  obtained  by 
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choosing  g  <  0  with  same  absolute  value.  The  parameter  h 
determines  heavy  tailedness.  As  h  increases,  heavy  tailedness 
increases  as  well.  For  the  case  h  =  0  the  g-and-h 
distribution, 

exp  (gZ,. .)  -1  .  „  ' 

=  — f^^-^  ,  (49) 

coincides  with  the  lognormal  distribution.  In  addition  to  the 
g-and-h  distributions  allowing  only  for  longer-tailed 
distributions  than  the  normal  distribution,  a  possible 
drawback  of  the  g-and-h  distributions  is  the  dependence  of  the 
spread  function  on  g.  The  skewness  depends  only  on  g  and 
increases  with  the  absolute  value  of  g,  but  kurtosis  increases 
with  h  and  with  the  absolute  value  of  g.  Table  3  cited  from 
Martinez  and  Iglewicz  (1984)  shows  the  values  of  g  and  h  that 
approximate  a  selected  set  of  well  known  distributions  in 
comparison  with  the  usual  coefficients  of  skewness  and 
kurtosis. 

The    nonnormal    data   were    generated   using   the  g-and-h 
distribution  as  the  following  steps: 

1.  For  the  jth  level  of  the  between-subjects  factor  Zj, 
an  n^  x  4  matrix  of  independent  normally  distributed  variates 
was  generated.  The  NORMAL  function  in  SAS  was  used  to 
generate  all  variates. 

2.  An  nj  x  4  matrix  X*   was  constructed  by  applying, 

*  =     j.exp  ( i2Z^/  /  2 )  .  (50) 


Table  3 

Coefficients  of  skewness  and  kurtosis  and  corresponding 
values  of  g  and  h  for  some  selected  distributions 


Distribution  skewness     kurtosis  g  h 


Uniform 

.00 

1 

.80 

.000 

-.244 

Normal 

.00 

3 

.00 

.000 

.000 

''-lO 

.00 

4. 

.00 

.000 

.058 

Logistic 

.00 

4, 

.20 

.000 

.065 

Double  Exponential 

.00 

6, 

.00 

.000 

.  109 

Cauchy 

.000 

.970 

Beta  (2,3) 

.29 

2, 

.36 

.  142 

-.101 

Weibull;  k  =  2 

.63 

3, 

,25 

.243 

-.042 

.89 

4. 

,20 

.303 

-.017 

1.41 

6. 

,00 

.502 

-.046 

Exponential 

2.00 

9. 

00 

.760 

-.098 

Lognorraal 

6.18 

113. 

94 

1.000 

.  .000 
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3.  The  rij  X  4  matrix  X*   was  transformed  to 

X.*   =  /i  +  djX.U'  ,  .  .  (51) 

where  fi,  d^,    and  U'   are  defined  as  in  the  second  step  of  the 
procedure  for  generating  multivariate  normal  data. 
Evaluating  the  Performance  of  Test  Procedures 

Type  I  error  rates  were  obtained  under  conditions  where 
the  population  mean  vector,  jx,  was  the  null  vector.  The  Type 
I  error  was  defined  the  probability  that  at  least  one  of  the 
pairwise  comparisons  was  statistically  significant.  Three 
different  types  of  power  were  computed:  (a)  the  average  per- 
pair  power  rate,  where  per-pair  power  is  the  probability  of 
detecting  a  non-zero  pairwise  difference  for  a  particular 
pairwise  comparison;  (b)  the  all-pairs  power  rate,  that  is, 
the  probability  of  detecting  all  non-zero  pairwise 
differences;  and  (c)  the  any-pairs  power  rate,  that  is,  the 
probability  of  detecting  at  least  one  non-zero  pairwise 
difference.  For  each  condition  5000  replications  were 
performed.  The  proportions  of  the  replications  that  resulted 
in  a  significant  test  at  a  =  .01,  a  =  .05,  and  a  =  .10  were 
reported.  Welsch's  (1977)  step-up  procedure  was  conducted 
only  at  a  =  .05. 

Summary 

Three  distribution  types  (g  =  0  and  h  =  0,  g  =  0  and  h  = 
.109,  and  g  =  0  and  h  =  .35),  two  levels  of  sphericity  of  the 
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common  v-c  matrix  (e  =  .96  and  .74),  three  levels  of  the 
degree  of  the  heterogeneity  of  the  v-c  matrices  (S^rSj  =  1:1/ 
1:2,  and  1:5),  two  levels  of  total  sample  size  (N  =  40,  and 
60),  three  levels  of  sample  size  ratio  (n^,  n^)  ^  28,  12;  20, 
20;  and  12,  28  for  N  =  40,  and  42,  18;  30,  30;  and  18,  42  for 
N  =  60),  and  a  configuration  of  null  means  and  three 
configuration  of  nonnull  means  combine  to  give  432 
experimental  conditions. 

Three  variations  of  the  Bonferroni  procedures  were 
included:  (a)  the  Bonferroni  (B),  (b)  the  raw  score  trimmed 
Bonferroni  (B^^),  and  (c)  the  trimmed  difference  score 
Bonferroni  (Bp^).  Six  variations  of  the  Shaffer's  modified 
seguentially  rejective  Bonferroni  (MSRB)  procedures  were 
included:  (a)  Shaffer's  MSRB  procedure  using  e-adjusted  test 
for  omnibus  test  (S(e)),  (b)  Shaffer's  MSRB  procedure  using  e- 
adjusted  test  for  omnibus  test  with  the  raw  score  trimmed  data 
(S(e)RT),  (c)  Shaffer's  MSRB  procedure  using  e-adjusted  test 
for  omnibus  test  with  the  trimmed  difference  score  (S(e)p^), 
(d)  Shaffer's  MSRB  procedure  using  the  trimmed  CIGA  test  for 
omnibus  test  (S(C)),  (e)  Shaffer's  raw  score  trimmed  MSRB 
procedure  using  the  trimmed  CIGA  test  for  omnibus  test 
(S(C)p^),  and  (f)  Shaffer's  trimmed  difference  score  MSRB 
procedure  using  the  trimmed  CIGA  test  for  omnibus  test 
(S(C)p^).  Two  variations  of  Welsch's  step-up  procedures  were 
included:  (a)  Welsch's  step-up  procedure  (W)  and  (b)  raw  score 
trimmed  Welsch  procedure  (W„) . 


CHAPTER  4  -  ' 

'      RESULTS  AND  DISCUSSION  > 

In  this  chapter  analyses  of  Type  I  error  rates  and  power 
are  presented  for  tests  conducted  at  a  =  .05.  The  pattern  of 
results  for  tests  conducted  at  a  =  .01  and  for  a  =  .10  was 
similar.  Results  are  reported  in  five  sections:  (a)  Type  I 
error  rates  for  a  completely  true  null  hypothesis,  (b)  Type  I 
error  rates  for  a  partially  true  null  hypothesis,  (c)  any-pair 
power,  (d)  average  per-pair  power,  and  (e)  all-pairs  power. 
Each  subsection  begins  with  a  table  of  percentiles  of  the 
distributions  of  Type  I  error  rates  or  power.  Then  a  summary 
of  the  analysis  of  variance  (ANOVA)  used  to  investigate  the 
effect  of  the  within-subjects  factor  and  the  between-subjects 
factors  on  Type  I  error  rates  or  power  is  presented.  The 
within-subjects  factor  was  test  (T)  and  the  between-subjects 
factors  were  distribution  of  the  generated  data  (D), 
sphericity  of  the  common  v-c  matrix  (e),  degree  of  v-c 
matrices  heteroscedasticity  (H^ii:^) ,  total  sample  size  (N), 
sample  size  arrangement  {n^/n^) ,  and  configuration  of  the 
means   (CM) . 

Because  many  of  the  factors  that  affect  Type  I  error 
rates  and  power  were  included  in  the  study,  each  ANOVA  yielded 
a  substantial  number  of  significant  effects.     To  compare  the 
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size  of  the  effects  isr  (Myers,  1979)  was  calculated  for  each 
effect.  A  mean  square  component  was  computed  for  each  effect 
m  each  ANOVA.  The  mean  square  component,  0^*^,  for  an  effect 
was  computed  using  the  formula  [ (MSF^  -  MSE)  /  x],  where  MSF- 
was  the  mean  square  for  the  given  effect,  MSE  was  the  between- 
subjects  mean  square  error  when  the  given  effect  was  between- 
subjects  effects  and  the  within-subjects  mean  square  error 
when  the  given  effect  was  the  within-subjects  effect  or  the 
interaction  of  the  within-subjects  factor  and  between-subjects 
factor(s),  and  x  was  the  number  of  levels  for  the  factors  not 
included  in  the  given  effect.  Negative  mean  square  components 
were  set  to  zero.  For  each  effect  with  a  non-negative  mean 
square  component,  u^"  was  computed  using  the  formula  6^*^  / 
(SMSF^  +  MSEi,  +  MSE^),  where  MSE^  is  the  between-subjects  mean 
square  error  and  MSE^^  is  the  within-subjects  mean  square 
error.  The  «^  values  were  used  to  select  significant  effects 
for  interpretation.  Only  effects  for  which  was  larger  than 
.05  were  selected.  This  criterion  is  arbitrary.  However, 
inspection  of  effects  with  in  the  range  (.01,  .05)  suggests 
these  effects  are  small  in  magnitude.  ;-^H 

Type  I  Error  Rates 

Completely  True  Null  Hypothesis 

Distributions  of   r  for  the   11   tests  are  summarized  in 
Table  4.     The  standard  error  of  these  estimated  Type  I  error 
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Table  4  .  • ' 

Distributions  of  Type  I  Error  Rates  for  the  Eleven  Tests: 
ComDletelv  True  Null  Hypothesis  and  a  =  .05 


Test 

Min 

10 

25 

50 

75 

90 

Max 

B 

.0234 

.0250 

.0278 

.0374 

.0412 

.0450 

.0472 

Brt 

.0222 

.0260 

.0292 

.0326 

.0346 

.0368 

.0408 

Bdt 

.0330 

.0380 

.0406 

.0450 

.0496 

.0544 

.0656 

W 

.0388 

.0448 

.0482 

.0544 

.0594 

.0656 

.0744 

.0360 

.0418 

.0450 

.0484 

.0528 

.0576 

.0664 

S(e) 

.0030 

.  0066 

.0204 

.0326 

.0422 

.0554 

.0762 

S(Ort 

.0038 

.  0090 

.0254 

.0332 

.0378 

.0490 

.  0596 

S(e)DT 

.0034 

.0066 

.0190 

.0252 

.0306 

.  0498 

.0668 

S(C) 

.0188 

.0236 

.0272 

.0354 

.0402 

.0448 

.0488 

S(C)rt 

.0176 

.0264 

.0310 

.0332 

.0354 

.0384 

.0428 

S(C)oT 

.0154 

.0190 

.0222 

.0256 

.0270 

.0294 

.0348 

Note.  B  =  Bonferroni;  B^^  =  raw  score  trimmed  Bonferroni;  Bp^ 
=  trimmed  difference  score  Bonferroni;  W  =  Welsch's  step-up 
procedure;  W^^  =  raw  score  trimmed  Welsch  step-up  procedure; 
S(e)  =  Shaffer's  Modified  Sequentially  Rejective  Bonferroni 
(MSRB)  procedure  using  an  e-adjusted  omnibus  test£  S(e)p^  = 
Shaffer's  raw  score  trimmed  MSRB  procedure  using  an  e-adjusted 
omnibus  te_st;  S{e)^j  =  Shaffer's  trimmed  difference  score  MSRB 
using  an  e-adjusted  omnibus  test;  S(C)  =  Shaffer's  Modified 
Sequentially  Rejective  Bonferroni  (MSRB)  procedure  using  a 
CIGA  omnibus  test;  S{C)^j  =  Shaffer's  raw  score  trimmed  MSRB 
procedure  using  a  CIGA  omnibus  test;  S(C)qj  =  Shaffer's 
trimmed  difference  score  MSRB  using  a  CIGA  omnibus  test. 
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rates  is  [a(  1-a) /5000  j"*^^.  If  the  population  Type  I  error 
rate  were  .05,  the  standard  error  would  be  .0031,  so  that  the 
rejection  region  for  an  upper-tailed  z  test  of  Hq:  a  =  .05  is 
.055  at  a  .05  significance  level.  As  would  be  expected  from 
the  theoretical  developments  underlying  the  B  and  S(C)  tests, 
using  the  upper-tailed  z  test  these  tests  do  not  result  in 
Type  I  error  rates  above  .05.  The  Bp^,  S{C)f^j,  and  S(C)p^ 
tests  also  share  the  property  and,  in  fact,  appear  to  be  more 
conservative  than  their  non-trimmed  counterparts.  As  expected 
from  theory  and  empirical  results,  the  S(e)  test  can  be 
liberal  in  some  conditions.  However,  the  test  is  not  too 
badly  liberal  and  trimming  tempers  the  liberal  tendency.  The 
Welsch  test  appears  to  have  the  strongest  liberal  tendency, 
presumably  because  the  critical  value  for  the  Welsch  test  has 
no  theoretical  support  when  multisample  sphericity  is 
violated.  In  the  trimmed  version,  Wp^,  the  liberal  tendency 
is  reduced.  By  Bradley's  (1978)  liberal  criterion  a  test  is 
robust  if  .5a  <  r  <  1.5a,  where  a  is  the  nominal  significance 
level  and  r  is  the  true  probability  of  Type  I  error  rate. 
None  of  the  tests  are  liberal  using  Bradley's  criterion. 

Selecting  among  the  tests  would  depend  on  one's  tolerance 
for  tests  that  can  be  somewhat  liberal  in  some  conditions.  To 
avoid  such  tests  and  to  avoid  low  Type  I  error  rates,  B 
appears  to  be  the  best  choice.  If  a  somewhat  liberal  test  can 
be  tolerated  B^^,    W  or  Wp^   appear  to  be  the  best  choice. 


A  3  (Distribution)  x  2  (Sphericity)  x  3  (V-C 
Heteroscedasticity )  x  3  (Sample  Size  Arrangement)  x  2  (Total 
Sample  Size)  x  11  (Test)  ANOVA,  with  repeated  measures  on  the 
test  factor  was  used  to  analyze  the  Type  I  error  rates  (r). 
Among  the  30  effects  involving  only  between-subjects  factors, 
the  main  effects  of  D,  n^/nj,  and  N,  and  the  n^/nj  x  S^:S2 
interaction  were  significant.  Among  the  31  effects  involving 
the  T  factor  and  interactions  of  test  with  the  between- 
subjects  factors,  the  main  effect  of  T,  all  5  two-way 
interactions,  and  9  higher  order  interactions  were 
significant.  Shown  in  Table  5  are  the  effects  that  accounted 
for  more  than  1%  of  the  total  variance. 

Effect  of  T.  Shown  in  Table  6  are  mean  values  of  z  for 
the  eleven  tests.  The  means  are  similar  to  the  medians  in 
Table  4  and  do  not  alter  the  interpretations  presented  in  the 
discussion  of  Table  4.  '  .4 

The  T  X  n^/n^   x  and  T  x  n-,/n-,    effects.     Means  for 

interpreting  the  T  x  n^/n^  x  S^iSj  interaction  are  presented 
in  Table  7.  Results  in  the  table  indicate  that  when  n^  =  n^, 
heterogeneity  of  the  v-c  matrices  has  little  effect  on  z. 
Based  on  Type  I  error  rates,  when  n^  =  n^,  the  best  procedures 
appear  to  be  W,  W^^,  and  B^j  because  they  maintain  z  nearest 
to  a.  When  E-,:E2  =  1:1,  n^/n^  appears  to  have  relatively 
little  effect  on  z.  However,  when  the  design  is  unbalanced, 
those  concerned  about  liberal  tests  might  avoid  W  and  use 
either  W^^   or  Bp^.      When  heterogeneous  v-c  matrices  are 


Table  5 

Percent  of  Variance  for  Type  I 
Error  Rates  for  Effects  that 
Accounted  for  at  least  1%  of 
the  variance;   Completely  True 
Null  Hypothesis  and  a  =  .05 


Effect 


(i) 
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T  X  n^/nj 

T  X  n^/nj   X  rSj 

T  X  D 

D 

T  X  e 


0.  623 
0. 178 
0.099 
0.023 
0.013 
0.013 


« 


Table  6 


Mean  Type  I  Error  Rates 
as  a  Function  of  Test: 
Completely  True  Null 
Hypothesis  and  a  =  .05 


Test  mean  r 


B 

0.035 

Brt 

0.032 

0.046 

W 

0.055 

^RT 

0.049 

S(e) 

0.033 

S(e)RT 

0.032 

S(Odt 

0.027 

S(C) 

0.035 

S(C)rt 

0.033 

0.025 

Note.  See  Table  4  for 
abbreviations . 


Table  7 


Mean  Type  I  Error  Rates  as  a  Function  of  V-C 
Heterogeneity,  Test,  and  Sample  Size 
Arrangement;  Completely  True  Null  Hypothesis 
and  g  =  .05 


S^rSg        Test  n-,<n2  n^yn^ 


B 

0 

.037 

0 

.036 

0. 

035 

Brt 

0 

.033 

0 

.031 

0. 

032 

Bdt 

0 

.048 

0 

.043 

0. 

046 

H 

0 

.058 

0 

.053 

0. 

054 

0 

.051 

0 

.  046 

0. 

050 

S(e) 

0 

.032 

0 

.  036 

0. 

031 

S(Ort 

0 

.032 

0 

.034 

0. 

031 

S(e)i,T 

0 

.027 

0 

.024 

0. 

026 

S(C) 

0 

.035 

0 

.036 

0. 

034 

S(C)pT 

0 

.033 

0 

.034 

0. 

033 

S(C)p, 

0 

.026 

0 

.024 

0. 

025 

Table  7  --  continued. 


Test 

n^<n2 

n^=n2 

n^  >n2 

B 

0.035 

0.035 

0.036 

Brt 

0 . 032 

0.030 

0.033 

0.044 

0.043 

0.050  • 

W 

0.054 

0.053 

0.056 

0  .  048 

0.046 

0.053 

S(8) 

0.016 

0.036 

0.047 

S(Ort 

0.017 

0.034 

0.045 

S(€)dt 

0.014 

0.024 

0.041 

S(C) 

0.036 

0.035 

0.033  ^ 

S(C)rt 

0.033 

0.034 

0.032 

0.025 

0.024 

0.026 

Table  7  —  continued. 


S^rSj         Test  n^<n2  ^^1=^2  n-,>n2 


B 

0.034 

0. 

035 

0 

.037 

0.032 

0. 

032 

0 

.032 

Bdt 

0  .  042 

0. 

045 

0 

.052 

W 

0.052 

0. 

053 

0 

.059 

Wrt 

0.047 

0. 

048 

0 

.054 

S(e) 

0.005 

0. 

035 

0 

.058 

S(Ort 

0.007 

0. 

034 

0 

.052 

S(Odt 

0.006 

0. 

025 

0 

.056 

S(C) 

0.035 

0. 

034 

0 

.033 

S(C)rt 

0.034 

0. 

033 

0 

.029 

0.025 

0. 

024 

0 

.026 

Note.  See  Table  4  for  abbreviations. 
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positively  paired  with  unequal  group  sizes  (S^zSj   *  1:1,  n-,  < 
^2)'    Type  I  error  rates  are  most  strongly  deflated  for  S(e), 
S(8)„^,    and  S(£)p^.      The  Bonferroni  and  Welsch  tests  show  a 
minor  degree  of  inflation.    The  S(C),  S(C)p^,    and  S(C)p^  tests 
show    no    deflation.       When    heterogeneous    v-c    matrices  are 
negatively  paired  with  unequal  group  sizes  (S^iSj       1:1,  n^  > 
^2)'    Type  I  error  rates  are  most  strongly  inflated  for  the 
S(e),  S(e)RT,   and  S(e)oT  tests.    The  B,  Bp^,    S(C),  S(C)„^,  S(C)o^ 
exhibit  little  or  no  inflation.    The  remaining  tests  (B^j,  W, 
and  Wp^)    exhibit  moderate  inflation.     If  heterogeneity  of  v-c 
matrices  is  expected,  those  concerned  about  liberal  tests  may 
want  to  use  B^^  or  Wp^   rather  than  W.     To  interpret  T  x  n^/n^ 
effect  the  means  in  Table  7  would  be  collapsed  over  values  of 
2^:22.      This  would  yield  means   similar  to  those  for  11^:11^  = 
1:2.     Consequently  the  T  x  n^/n^   effect  does  not  provide  any 
new  information  about  the  effect  of  T  and  n^/nj   on  r. 
Partially  True  Null  Hypothesis 

Distributions  of  r  for  partially  true  null  hypotheses  are 
summarized  in  Table  8  for  the  11  tests.  Because  all  of  the 
tests  are  designed  to  control  the  farailywise  Type  I  error  rate 
for  a  completely  true  null  hypothesis,  they  have  very  low  Type 
I  error  rates  for  a  partially  true  null  hypothesis.  The 
procedures  using  trimmed  data  appear  to  be  more  conservative 
than  their  non-trimmed  counterparts.  The  W  and  Wp^  tests  have 
Type  I  error  rates  closest  to  the  nominal  significance  level. 
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Table  8 

Distributions   of   Type   I   Error  Rates   for  the  Eleven  Tests: 


Partially  True  Null  Hypothesis  and  a  =  .05 


Test  Min  10  25  50  75  90  Max 


B  .0028  .0052  .0072  .0096  .0150  .0174  .0206 

B^j  .0036  .0050  .0060  .0086  .0128  .0148  .0180 

.0052  .0070  .0086  .0116  .0168  .0200  .0252 

W  .0082  .0122  .0174  .0234  .0374  .0428  .0488 

Wrt  .0092  .0122  .0148  .0202  .0328  .0364  .0438 

S(8)  .0012  .0046  .0090  .0138  .0218  .0292  .0380 

S(e)RT  .0024  .0062  .0090  .0130  .0204  .0258  .0326 

S(e)DT  .0028  .0062  .0098  .0142  .0212  .0256  .0386 

S(C)  .0022  .0052  .0096  .0140  .0238  .0284  .0340 

S(C)p^  .0030  .0064  .0088  .0144  .0208  .0236  .0302 

S(C)pT  .0040  .0068  .0094  .0140  .0214  .0248  .0328 


Note.  See  Table  4  for  abbreviations. 
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A  2  (Configuration  of  Means)  x  3  (Distribution)  x  2 
(Sphericity)  x  3  (V-C  Heteroscedasticity )  x  3  (Sample  Size 
Arrangement)  x  2  (Total  Sample  Size)  x  11  (Test)  ANOVA,  with 
repeated  measures  on  the  test  factor  was  used  to  analyze  the 
rs.  The  two  configurations  of  means  for  which  the  null 
hypothesis  is  partially  true  are  the  minimum  range  and  the 
maximum  range  configurations.  Among  the  62  effects  involving 
only  between-subjects  factor,  all  main  effects  except  v-c 
heteroscedasticity,  4  two-way  interactions,  and  2  higher  order 
interactions  were  significant.  Among  the  63  effects  involving 
the  T  factor  and  interactions  of  test  with  the  between- 
subjects  factors,  the  main  effect  of  T,  all  5  two-way 
interactions,  and  20  higher  order  interactions  were 
significant.  Shown  in  Table  9  are  the  effects  that  accounted 
for  more  than  1%  of  the  total  variance.  •  - 

The  T.  CM.  and  T  x  CM  effects.  Shown  in  Table  10  are 
mean  values  of  r  for  the  11  tests.  The  means  are  similar  to 
the  medians  in  Table  8  and  do  not  alter  the  interpretations 
presented  in  the  discussion  of  Table  8.  Means  for 
interpreting  the  T  x  CM  interaction  are  presented  in  Table  11. 
Results  in  the  table  show  that  Type  I  error  rates  are  higher 
for  the  minimum  range  configuration  than  for  the  maximum  range 
configuration.  This  result  occurs  because  the  minimum  range 
configuration  has  two  true  pairwise  comparisons  and  the 
maximum  range  configuration  has  one  true  pairwise  comparison. 


Table  9 

Percent  of  Variance  for  Type  I 
error  rates  for  Effects  that 
Accounted  for  at  least  1%  of 
the  Variance:  Partially  True 
Null  Hypothesis  and  a  =  .05 


Effect  0)2 


T  0.618 

T  X  CM  0.088 

CM  0.085 

T  X  D  0.061 

T  X  n^/ng  0.039 
T  X  n^/nj   X  S^rSj  0.024 

D  0.022 

T  X  E  0.011 


Table  10 


Mean  Type  I  error  rates 
as  a  Function  of  Test; 
Partially  True  Null 
Hypothesis  and  a  =  .05 


Test  mean  r 


B 

0.011 

0.009 

Bdt 

0.013 

W 

0.026 

^RT 

0.024 

S(£) 

0.016 

0.015 

S(e)BT 

0.016 

S(C) 

0.016 

S(C)rt 

0.015 

S(C)bt 

0.015 

Note.  See  Table  4  for 
abbreviations . 


Table  11 


Mean  Type  I  error  rates  as  a  Function 
of  Test  and  the  Configuration  of 
Means:  Partially  True  Null  Hypothesis 
and  g  =  .05 


Minimum 

Maximum 

Test 

range 

range 

B 

0.014 

0.007 

Brt 

0.013 

0.006 

0.017 

0.009 

W 

0.035 

0.017 

Wrt 

0.032 

0.015 

S(e) 

0.021 

0.010 

S(Ort 

0.020 

0.009 

S(£)dt 

0.021 

0.010 

S(C) 

0.022 

0.010 

S(C)rt 

0.020 

0.009 

S(C)oT 

0.021 

0.010 

Note.  See  Table  4  for  abbreviations. 
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For  both  configurations,  Type  I  error  rates  are  closest  to  .05 
for  the  W  and  W^^  tests. 

The  T  X  D  effect.  In  Table  12,  mean  Type  I  error  rates 
are  reported  for  each  distribution.  All  test  procedures  have 
lower  Type  I  error  rates  when  the  distribution  has  long  tails. 
As  kurtosis  increases  the  Type  I  error  rates  for  the 
procedures  using  non-trimmed  data  tend  to  decline  more  rapidly 
than  do  the  Type  I  error  rates  for  procedures  using  trimmed 
data.  Among  the  11  tests,  the  W  and  tests  tend  to  have 
Type  I  error  rates  closest  to  .05.  For  a  normal  distribution, 
W  has  the  Type  I  error  rate  that  is  closest  to  .05.  As 
kurtosis  increases  the  Type  I  error  rates  for  W  and  W,,^.  become 
more  similar. 

Power 

In  this  section  power  results  are  presented  in  three 
subsections:  any-pair  power,  average  per-pair  power,  and  all- 
pairs  power.  In  each  subsection,  percentiles  of  the 
distribution  of  power  are  presented  for  each  test.  Then  the 
results  of  a  3  (Configuration  of  Means)  x  3  (Distribution)  x 
2  (Sphericity)  x  3  (V-C  Heteroscedasticity )  x  3  (Sample  Size 
Arrangement)  x  2  (Total  Sample  Size)  x  11  (Test)  ANOVA,  with 
repeated  measures  on  the  test  factor  are  presented. 
Any-Pair  Power 

Distributions   of   any-pair  power   for   the    11   tests  are 
summarized  in  Table  13.     The  W  and  W^^  tests  have  higher  any- 


Table  12 


Mean  Type  I  error  rates  as  a  Function  of 
Test  and  Distribution;  Partially  True 
Null  Hypothesis 


Test 

h=0 

h=. 109 

h=.35 

B 

0.013 

0.012 

0.008 

Brt 

0.011 

0.009 

0.008 

Bdt 

0.014 

0.013 

0.011 

W 

0.032 

0.028 

0.019 

Wot 
RT 

0.027 

0  .  024 

0 . 020 

S(e) 

0.021 

0.017 

0.009 

S(Ort 

0.017 

0.015 

0.012 

S(e)DT 

0.018 

0.016 

0.012 

S(C) 

0.021 

0.018 

0.009 

0.017 

0.015 

0.012 

S(C)dt 

0.018 

0.016 

0.011 

Note.  See  Table  4  for  abbreviations. 
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Table  13  .  . 

Distributions  of  Any-Pair  Power  for  the  Eleven  Tests  at  a  = 
.05 

Test  Min         10  25  50  75  90  Max 


B  .078  .103  .130  .269  .361  .443  .699 

B„  .100  .146  .178  .222  .282  .341  .541 

B^j  .126  .164  .203  .273  .339  .402  .621 

W  .123  .147  .175  .327  .425  .505  .777 

.152  .201  .232  .290  .346  .410  .633 

S(£)  .032  .088  .110  .260  .337  .432  .486 

S(e)RT  .094  .165  .188  .246  .288  .315  .368 

S(c)pT  .085  .145  .167  .239  .280  .316  .377 

S(C)  .077  .097  .111  .274  .361  .450  .629 

S(C)rt  .114  .168  .196  .253  .301  .348  .512 

S(C)oT  .102  .144  .166  .238  .289  .335  .481 


Note.  See  Table  4  for  abbreviations. 


78 

pair  power  than  the  other  tests,  and  thus  these  results  favor 
the  use  of  the  W  and  W^^  tests.  However  these  tests,  along 
with  Bpy  had  some  tendency  to  be  liberal  when  the  null 
hypothesis  was  completely  true.  Based  on  the  results  in  Table 
13,  those  who  want  to  avoid  a  liberal  test  should  select  the 
B  or  S(C)  test.  Among  the  62  effects  involving  only  between- 
subjects  factors,  all  main  effects  except  v-c 
heteroscedasticity ,  9  two-way  interactions,  and  7  higher  order 
interactions  were  significant.  Among  the  63  effects 
involvingthe  T  factor  and  interactions  of  test  with  the 
between-subjects  factors,  the  main  effect  of  T,  all  6  two-way 
interactions,  and  34  higher  order  interactions  were 
significant.  Shown  in  Table  14  are  the  effects  that  accounted 
for  more  than  1%  of  the  total  variance. 

The  T.  D,  and  T  x  D  effects.  Means  of  any-pair  power 
rates  for  interpreting  the  T  x  D  interaction  are  presented  in 
Table  15.  Results  in  the  table  indicate  that  all  eleven  tests 
have  less  power  when  the  data  are  generated  from  long-tailed 
distributions.  Tests  using  non-trimmed  data  have  high  power 
when  the  distribution  is  normal.  The  tests  using  trimmed  data 
have  about  10  to  25  percent  less  power  than  their  non-trimmed 
counterparts.  The  W  test  has  the  highest  any-pair  power  when 
the  distribution  is  normal.  The  W  test  also  has  the  highest 
power  when  the  distribution  is  slightly  kurtotic  (h  =  .109), 
but  Wp^,  B,  B^j,  and  S(C)  perform  well,  also.  In  the  very 
long-tailed  non-normal  distribution  (h  =  .35),  W^,  and  B..  have 


Table  14 


Percent  of  Variance  of  Any-Pair 
Power  for  Effects  that  Accounted 
for  at  least  1%  of  the  Variance; 
g  =  .05 


Effect  ©2 


T  X  D  0.234 

T  0.200 

T  X  n-j/ng  0  . 138 

D  0.132 

T  X  n^/nj  X  S^rSj  0.094 

T  X  CM  X  e  0.058 

T  X  CM  0.024 

T  X  e  0.018 

T  X  S^rSj  0.013 

T  X  n^/nj  X  D  0.012 

CM  X  e  0.010 


Table  15 


Mean  of  Any-Pair  Power  as  a  Function  of 
Test  and  Distribution:  a  =  .05 


Test 

h=0 

h=. 109 

h=.35 

B 

0 . 396 

0.289 

0.117 

0.288 

0.249 

0.171 

Bdt 

0.359 

0.296 

0. 189 

W 

0.468 

0.  355 

0.165 

0.360 

0.316 

0.226 

S(8) 

0.381 

0.268 

0.093 

S(e)RT 

0.290 

0.252 

0. 175 

S(e)oT 

0.292 

0.243 

0. 153 

S(C) 

0.408 

0.291 

0  .  105 

0.311 

0.270 

0. 188 

S(C)b, 

0.303 

0.251 

0. 156 

Note.  See  Table  4  for  abbreviations. 
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the  highest  powers.  The  procedures  using  non-triituned  data 
have  low  any-pair  power  when  h  =  . 35 . 

The  T  X  n^/n2   and  T  x  n-i/n,   x  S^iSo    effects.     Means  for 
interpreting  the  T  x  n^/nj    x  S^iSj    interaction  are  presented 
in  Table  16.     Based  on  any-pair  power,  when  n^   =  nj,   the  best 
procedures  appear  to  be  W,  Wp^,    and  Bp^.       However,  the  W  test 
had  slightly  elevated  Type  I  error  rates  when  n<,   =        and  the 
null  hypothesis  was  completely  true.     When  E^:S2   =  1:1,  n^/nj 
appears  to  have  relatively  little  effect  on  any-pair  power. 
When  heterogeneous  v-c  matrices  are  positively  paired  with 
unequal  group  sizes    (S^rSj    *   1:1,    n^    <   n^) ,    any-pair  power 
rates  for  Shaffer's  procedures  using  the  e -adjusted  omnibus  F 
test  decline;   power  rates  for  the  remaining  tests  increase. 
For  all  tests  the  degree  of  deflation  or  inflation  increases 
with  increases  in  the  degree  of  v-c  matrices  heterogeneity. 
When  heterogeneous  v-c  matrices   are  negatively  paired  with 
unequal  group  sizes    {Z^:^^    *   ^-l'  >   n^) ,    any-pair  power 

rates  for  all  tests  except  Shaffer's  procedures  using  the  e- 
adjusted  omnibus  F  test  decline,  with  the  degree  of  deflation 
increasing  with  increases  in  the  degree  of  v-c  matrices 
heterogeneity.  Shaffer's  procedures  following  an  e-adjusted 
omnibus  F  test  exhibit  no  deflation.  The  W  and  W^^  tests  have 
the  highest  power  for  both  positive  and  negative  pairing; 
Shaffer's  procedures  following  e-adjusted  F  test  have  very  low 
power  in  positive  pairing. 


Table  16 


Mean  of  Any-Pair  Power  as  a  Function  of  V-C 
Heterogeneity,  Test,  and  Sample  Size;  a  =  .05 


2^:22  Test  n^<n2  '^i~'^2  n^>n2 


B 

0 

.260 

0 

.267 

0 

.261 

Brt 

0 

.229 

0 

.234 

0 

.230 

Bdt 

0 

.277 

0 

.  277 

0 

.276 

W 

0 

.  323 

0 

.323 

0 

.325 

0 

.296 

0 

.293 

0 

.296 

S(e) 

0 

.254 

0 

.271 

0 

.256 

S(Ort 

0 

.250 

0 

.261 

0 

.251 

S(e)DT 

0 

.239 

0 

.235 

0 

.239 

S(C) 

0 

.262 

0 

.268 

0 

.263 

S(C)rt 

0 

.249 

0 

.259 

0 

.250 

0 

.233 

0 

.233 

0 

.233 

Table  16  --  continued. 


S^rSj  Test  n^<n2  '^i'^'^2  n^>n2 


1  •  "? 

D 

U  .  jUZ 

n    0  c; 
U  .  ^  DO 

n    o  o  o 
U  .  z  z  y 

Brt 

n   o  T  o 

n    o  o  o 

u .  ly D 

^DT 

U  .  J  i  4 

A      O  T  C 

0.245 

W 

U  .  io  6 

n    o  o  o 

A      O  A  >1 

0.294 

"rt 

0.339 

0.292 

0.263 

S(e) 

0.214 

0.269 

0.272 

0.219 

0.258 

0.254 

S(e)BT 

0.203 

0.233 

0.260 

S(C) 

0.303 

0.266 

0.229 

S(C)rt 

0.296 

0.255 

0.211 

0.274 

0.230 

0.  199 

Table  16  --  continued. 


S^rSj  Test  n^<n2  '^i'='^2  n^>n2 


0 

o  o  t 

n  9  R 

U  •  Z  D  3 

n 

u  . 

9  0"? 
U  J 

n 

°RT 

J  -J  *r 

n    9  9  0 

U  .  Z  Z  7 

u  . 

10  0 

n 

u  • 

7  A 

U  •      /  O 

n 

u  . 

9  1  R 
Z  1  O 

W 

0. 

419 

0.326 

0. 

268 

0. 

401 

0  .  293 

0. 

232 

S(e) 

0. 

155 

0.268 

0. 

270 

0. 

166 

0.254 

0. 

238 

S(e)DT 

0. 

158 

0.232 

0. 

265 

S(C) 

0. 

358 

0.264 

0. 

199 

S(C)rt 

0. 

362 

0.249 

0. 

177 

0. 

335 

0.226 

0. 

170 

Note.  See  Table  4  for  abbreviations. 
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The  T  X  CM  X  e  effect.  Means  for  interpreting  the  T  x  CM 
X  e  interaction  are  presented  in  Table  17.  For  all 
combinations  of  configurations  and  e ,  W  has  the  highest  any- 
pair  power,  followed  by  W^^.  The  results  in  Table  17  indicate 
that  (a)  the  impact  of  mean  configuration  on  power  is  smaller 
for  the  Shaffer  tests  than  it  is  for  the  Welsch  and  Bonferroni 
tests  and  (b)  the  impact  of  mean  configuration  on  power 
depends  on  e.  With  respect  to  (b)  when  e  =  .96,  power  is 
highest  for  the  maximum  range  configuration;  when  e  =  .75, 
power  is  highest  for  the  minimum  range  configuration.  The 
variation  in  the  relationship  between  mean  configuration  and 
power  as  a  function  of  e  is  due  to  the  impact  of 

max   '-J^   .^2\ 

on  power.  Because  of  the  means  and  dispersion  matrices,  used 
in  simulating  the  data,  when  e  =  .96  the  ratio  in  eguation 
(52)  is  largest  for  the  maximum  range  configuration  and 
smallest  for  the  minimum  range  configuration.  When  e  =  .75, 
the  ratio  in  (52)  is  largest  for  the  minimum  range 
configuration  and  smallest  for  the  maximum  range 
configuration.  The  smaller  impact  of  mean  configuration  on 
power  for  the  Shaffer  tests  is  due  to  the  impact  of  the 
omnibus  test  on  the  power  of  these  procedures.  For  the 
minimum  range  configuration  any-pair  power  for  the  Bonferroni 
and  Welsch  tests  increases  as  e  changes  from  .96  to  .75. 
However,  any-pair  powers  for  these  tests  decrease  as  e  changes 


Table  17 


Mean  of  Any-Pair  Power  as  a  Function  of  Sphericity, 
Test,  and  the  Configuration  of  Means:  a  =  .05 

minimum      maximum  equally 
Sphericity    Test  range  range  spaced 


B 

0.251 

0.299 

0.278 

Brt 

0.219 

0.261 

0.245 

Bdt 

0.265 

0.306 

0.289 

W 

0.301 

0.347 

0.329 

Wrt 

0.274 

0.316 

0.300 

S(e) 

0.260 

0.276 

0.270 

S(e)RT 

0.244 

0.260 

0.254 

S(8)oT 

0.233 

0.245 

0.241 

S(C)  . 

0.284 

0.301 

0.295 

0.265 

0.282 

0.276 

S(C)dt 

0.243 

0.254 

0.249 

Table  17  —  continued. 


minimum      maximum  equally 
Sphericity    Test  range  range  spaced 


Bdt 
W 

S(e) 

S(e)RT 
S(e)DT 
S(C) 

S(C)rt 
S(C)., 


0.351 
0.297 
0.364 
0.433 
0.381 
0.242 
0.238 
0.236 
0.255 
0.249 
0.240 


0.201 
0  .  187 
0.219 
0.265 
0.252 
0.217 
0.216 
0.208 
0.235 
0.231 
0.215 


0.226 
0.206 
0.246 
0.301 
0.280 
0.220 
0.221 
0.215 
0.238 
0.235 
0.221 


Note.  See  Table  4  for  abbreviations. 
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from    .96   to    .75    for   the  maximum   range   and   equally  spaced 
configurations.    These  results  reflect  changes  in  the  ratio  in 
equation  ( 52 ) . 
Average  Per-Pair  Power 

Distributions  of  average  per-pair  power  for  the  11  tests 
are  summarized  in  Table  18.  The  W  and  Wp^  tests  have  the 
highest  average  per-pair  power.  The  Bonferroni  procedures 
have  low  average  per-pair  power.  ' 

Among  the  62  effects  involving  only  between-subjects 
factors,  all  6  main  effects,  10  two-way  interactions,  and  6 
higher  order  interactions  were  significant  at  a  =  .05.  Among 
the  63  effects  involving  the  T  factor  and  interactions  of  test 
with  the  between-subjects  factors,  the  main  effect  of  T,  all 
6  two-way  interactions  with  test,  and  32  higher  order 
interactions  were  significant.  The  effects  that  were 
significant  were  almost  same  as  the  significant  effects  for 
any-pair  power.  Shown  in  Table  19  are  the  effects  that 
accounted  for  more  than  1%  of  the  total  variance. 

The  T.  T  X  CM.  and  T  x  CM  x  e  effects.  Means  for 
interpreting  the  T  x  CM  x  e  interaction  are  presented  in  Table 
20.  For  all  combinations  of  configurations  and  e,  W  has  the 
highest  average  per-pair  power,  followed  by  W^^.  As  did  the 
results  for  any-pair  power,  the  results  in  table  indicate  that 
the  impact  of  mean  configuration  on  power  is  smaller  for  the 
Shaffer  tests  than  it  is  for  the  Welsch  and  Bonferroni  tests. 
For  both  e  =  .96  and  .75  power  is  largest  for  the  minimum 
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Table  18 

Distributions  of  Average  Per-Pair  Power  for  the  Eleven  Tests 


at  g  -  .05 


Test 

Min 

10 

25 

50 

75 

90 

Max 

B 

.020 

.025 

.036 

.074 

.111 

.  150 

.308 

.024 

.038 

.048 

.064 

.085 

.  Ill 

.218 

Bdt 

.029 

.040 

.055 

.076 

.  102 

.  133 

.  253 

W 

.047 

.064 

.098 

.  170 

.249 

.396 

.723 

Wrt 

.056 

.085 

.112 

.  151 

.208 

.316 

.566 

S(8) 

.014 

.029 

.  036 

.098 

.  134 

.  177 

.220 

S(e)RT 

.037 

.053 

.066 

.083 

.  105 

.  126 

.  154 

S(e)DT 

.028 

.045 

.058 

.083 

.  106 

.  130 

.  156 

r>  o  "3 

AO! 
.  U  J  1 

.040 

.099 

.139 

.  198 

.317 

S(C)rt 

.030 

.051 

.066 

.086 

.114 

.  138 

.235 

.026 

.044 

.059 

.082 

.110 

.  139 

.242 

Note.  See 

Table 

4  for 

abbreviations . 

i 

•>" 

Table  19 


Per-Pair  Power  for 

Effects  that 

Accounted  for  at  least  1%  of  the 

Variance:  a  =  .05 

Effect 

til 

T 

0.509 

T  X  CM 

0. 134 

T  X  D 

0.113 

T  X  CM  X  e 

0.050 

D 

0.040 

T   X  8 

0.032 

CM 

0.021 

T  X  n^/nj 

0.021 

T  X  CM  X  D 

0.020 

T  X  n^/nj    X  S^iSj 

0.015 

Table  20 


Mean  of  Average  Per-Pair  Power  as  a  Function  of 
Sphericity,  Test,  and  the  Configuration  of  Means 
g  =  .05 


minimum      maximum  equally 
Sphericity    Test  range  range  spaced 


B 

0  . 

097 

Brt 

0. 

079 

Bdt 

0. 

094 

W 

0. 

214 

0. 

189 

S(e) 

0. 

120 

S(Ort 

0. 

105 

S(Odt 

0. 

102 

S(C) 

0. 

129 

0. 

112 

0. 

107 

0.085  0.068 

0.071  0.057 

0.083  0.067 

0.189  0.111 

0.166  0.098 

0.096  0.081 

0.085  0.072 

0.081  0.069 

0.104  0.088 

0.092  0.077 

0.085  0.073 


Table  20  --  continued. 


minimum      maximum  equally 
Sphericity    Test  range         range  spaced 


.75 


B 

0.130 

0.067 

0  .  061 

n  1 

0. 105 

0.058 

0.052 

U  .  U  0  D 

U  .  U  0 1 

w 

0.377 

0.  176 

0.  121 

0.319 

0.161 

0.  107 

S(e) 

0. 121 

0.087 

0.075 

S(Ort 

0. 108 

0.080 

0.069 

S(e)DT 

0.  108 

0  .  078 

0.068 

S(C) 

0. 128 

0.093 

0.080 

S(C)rt 

0.114 

0.084 

0.073 

S(C)i,T 

0. 113 

0.081 

0.071 

Note.     See  Table  4  for  abbreviation. 
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range  configuration  and  smallest  for  the  equally  spaced 
configuration.  The  effect  of  mean  configuration  is  due  to  the 
magnitude  of  the  differences  between  nonnull  means  in  the 
three  types  of  configurations.  For  the  minimum  range 
configuration,  the  four  differences  between  nonnull  means  have 
same  size.  For  the  maximum  range  configuration  and  the 
equally  spaced  configuration,  the  difference  between  the 
largest  and  smallest  means  is  large,  but  the  other  four 
differences  are  moderate  in  comparison  to  the  difference  for 
the  minimum  range  configuration.  For  the  minimum  range 
configuration,  average  per-pair  power  tends  to  increase  for 
all  tests  as  e  changes  from  .96  to  .75.  The  largest  increases 
are  for  the  W  and  W^^  tests.  For  all  tests  the  effect  of  e  on 
power  is  small  for  the  maximum  range  and  equally  spaced  mean 
configurations.  This  pattern  of  results  is  predictable  from 
the  average  value  of 

,  =  (53) 

for  the  non-null  contrasts  for  each  combination  of  e  and  mean 
configuration.  As  e  changes  from  .96  to  .75,  the  change  in 
this  average  is  larger  for  the  minimum  range  configuration, 
than  it  is  for  the  other  two  configurations. 

The  T  X  D  effect.  Means  for  interpreting  the  T  x  D 
interaction  are  presented  in  Table  21.  The  T  x  D  interaction 
for  average  per-pair  power  is  similar  to  the  T  x  D  interaction 
for  any-pair  power.    The  results  in  the  Table  21  indicate  that 


Table  21 

Mean  of  Average  Per-Pair  Power  as  a 
Function  of  Test  and  Distribution;  a  = 


.05 

Test 

h=0 

h=. 109 

h=.35 

B 

0. 133 

0.090 

0.032 

Brt 

0.089 

0.074 

0.048 

0. 110 

0.087 

0.052 

W 

0.289 

0.213 

0.093 

Wrt 

0.212 

0.  182 

0.126 

S(e) 

0.156 

0. 103 

0.032 

S(e)RT 

0. 109 

0.091 

0.060 

S(e)DT 

0.113 

0.089 

0.051 

S(C) 

0. 165 

0.111 

0.036 

S(C)rt 

0.115 

0.097 

0.064 

0. 118 

0.093 

0.053 

Note.  See  Table  4  for  abbreviations. 
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all  eleven  tests  have  less  power  when  the  data  are  generated 
from  long-tailed  distributions.     The  W  test  has  the  highest 
average  per-pair  power  in  the  normal  distribution.     The  tests 
using  non-trimmed  data  have  high  power  when  the  distribution 
is  normal.     The  tests  using  trimmed  data  have  about  25  to  40% 
less   power   than   their   counterparts.      The   W   test   has  the 
highest  power  when  h  =  .109,  and  the  trimmed  version,  W^^,  has 
the  next  highest  power.    In  the  very  long-tailed  distribution 
(h  =   .35),  Wp^    has  the  highest  average  per-pair  power  and  W 
has  the  next  highest  power. 
All-Pairs  Power 

Distribution  of  all-pairs  power  for  the  11  tests  are 
summarized  in  Table  22.  All  eleven  tests  have  low  all-pairs 
power.  However,  power  is  highest  for  the  W  and  Wp^  tests. 
Because  power  is  zero  for  so  many  cells,  the  results  of  the 
ANOVA  for  the  full  set  of  the  data  will  not  be  reported.  The 
(I)  values  were  calculated  and  suggest  a  strong  T  x  CM 
interaction.  Means  for  interpreting  the  interaction  are 
presented  in  Table  23.  The  results  indicate  power  is  very  low 
except  for  the  W  and  W^^  tests  under  the  minimum  range 
configuration. 

For  the  W  and  W,,^  tests  under  the  minimum  range 
configuration,  a  3  (Distribution)  x  2  (Sphericity)  x  3  (V-C 
Heteroscedasticity)  x  3  (Sample  Size  Arrangement)  x  2  (Total 
Sample  Size)  x  2  (Test)  ANOVA,  with  repeated  measures  on  the 
test  factor  was  conducted.     Among  the  30  effects  involving 
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Table  22 

Distributions  of  All-Pairs  Power  for  the  Eleven  Tests  at  a 


=  .05 


Test 

Min 

10 

25 

50 

75 

90 

Max 

B 

0 

0 

0 

0 

.0018 

.0122 

.0476 

Brt 

0 

0 

0 

0 

.0020 

.0060 

.0228 

0 

0 

0 

0 

.0018 

.0062 

.0250 

W 

0 

0 

0 

.0096 

.  1216 

.3218 

.6622 

"rt 

0 

n 

n 

.  U  U  D  *r 

10  0  0 

.  1  z  ^  z 

O  "3  Q  y1 

.4940 

S(e) 

0 

0 

0 

.0004 

.  0068 

.0272 

.0844 

S(«)rt 

0 

0 

0 

.  0004 

.  0060 

.0154 

.0496 

0 

0 

0 

.0006 

.0060 

.0160 

.0472 

S(C) 

0 

0 

0 

.0004 

.0068 

.0272 

.0864 

S(C)pT 

0 

0 

0 

.  0004 

.0060 

.0154 

.0498 

0 

0 

0 

.  0006 

.0064 

.0160 

.0528 

Note.  See  Table  4  for  abbreviations. 


Table  23 


Mean  of  All-Pairs  Power  as  a  Function  of 
Test  and  the  Configuration  of  Means:  a  = 
.05 

minimum      maximum  equally- 
Test  range  range  spaced 


n 
D 

•  u  u  y  ^  i 

.  U  U  U  J  0 

. UUUUU 

.00503 

.00015 

.00000 

Bdt 

.00521 

.00021 

.00000 

W 

.23246 

.01151 

.00001 

Wrt 

.18933 

. 00774 

.00000 

S(8) 

.02049 

. 00200 

.00005 

S(c)rt 

.01291 

.00102 

.00002 

S(e)DT 

.01256 

.00135 

.00009 

S(C) 

.02059 

.00200 

.00005 

S.(C)rt 

.01292 

.00102 

.00002 

.01293 

.00136 

.00009 

Note.  See  Table  4  for  abbreviations. 
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only  between-subjects  factors,  all  5  main  effects,  5  two-way 
interactions,  and  3  higher  order  interactions  were 
significant.  Among  the  31  effects  involving  the  T  factor  and 
interactions  of  test  with  the  between-subjects  factors,  the 
main  effect  of  T,  4  two-way  interactions,  and  6  higher  order 
interactions  were  significant.  Shown  in  Table  24  are  the 
effects  that  accounted  for  more  than  1%  of  the  total  variance. 
"■  ■  ■  Effect  of  e  .  Means  for  interpreting  the  effect  of  e  are 
presented  in  Table  25.  The  all-pairs  power  for  the  W  and  Wp^ 
tests  are  higher  when  e  =  .75  than  when  e  =  .96  for  the 
minimum  range  configuration.  The  effect  of  e  is  due  to 
changes  in  the  average  value  of  eguation  (53)  when  e  changes 
from  .96  to  .75.  Because  of  means  and  v-c  matrices  selected 
for  this  study,  the  average  value  of  eguation  (53)  increases 
when  e  changes  from  .96  to  .75.  ^  •• 

The  T,  D,  and  T  x  D  effects.  Means  for  interpreting  the 
T  X  D  interaction  are  presented  in  Table  26.  Results  in  the 
table  indicate  that  both  of  the  tests  have  smaller  power  when 
the  data  are  generated  from  long-tailed  distributions.  As 
kurtosis  increases  the  power  rates  for  the  W  decline  more 
rapidly  than  do  the  power  rates  for  the  W^^.  The  W  test  has 
better  power  when  the  distribution  is  normal  (h  =  0)  and 
slightly  kurtotic  (h  =  .109).  The  W^^  test  has  better  power 
when  h  =  .35. 


Table  24 


Percent  of  Variance  for  All-Pairs 
Power  for  Effects  that  Accounted 
for  at  least  1%  of  the  Variance: 
g  =  .05 


Effect  «^ 


e  0.338 

D  0.322 

T  X  D  0.097 

T  0.053 

rii/n2  0.049 

D  X  e  0.045 

n^/nj   X  SitSj  0.036 

T  X  e  0.012 

T  X  D  X  e  0.012 


Note.  Only  W  and  Wp^  tests  are 
included  in  ANOVA. 
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Table  25 

Mean  All-Pairs  Power 
for  Sphericity:  a  - 
.05 


8  =  .96 


e  =  .  75 


134 


288 


Table  26 


Mean  of  All-Pairs  Power  as  a  Function 
of  Test  and  Distribution:  a  =  .05 


Test  h=0  h=.109  h=.35 


W  .351  .250  .097 

Wrt  .240  .199  .129 


Note.  See  Table  4  for  abbreviations. 
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DISCUSSION  AND  CONCLUSIONS 

*      •  '  <     ,  -' 

This  chapter  is  divided  into  three  sections:  (a)  findings 
of  related  research,  (b)  conclusions,  and  (c)  suggestions  for 
future  research. 

Findings  of  Related  Research 
Effect  of  Non-sphericity 

In  an  investigation  of  multiple  comparison  procedures  for 
a  design  with  a  single  within-subjects  factor.  Maxwell  (1980) 
found  that  the  Bonferroni  procedure  became  conservative  when 
sphericity  assumption  was  violated.     Keselman,  Keselman,  and 
Shaffer  (1991)  studied  the  split  plot  design  with  one  between- 
subjects   factor  and  one  within-subjects   factor.      They  used 
test  statistic  in  eguation  (3),  a  test  statistic  that  does  not 
require  the  assumption  of  equal  v-c  matrices.     The  critical 
values     studied    were    the    Bonferroni,     Studentized  range, 
Studentized   maximum   modulus,    and   Cochran    critical  values. 
Keselman,   Keselman,   and  Shaffer  found  that  the  Type  I  error 
rates  decreased  as  e  decreased  for  both  normal  and  nonnormal 
data  regardless  of  the  degree  of  heteroscedasticity .  Keselman 
(1994)  investigated  28  pairwise  multiple  comparison  procedures 
for  comparing  repeated  measures  means  in  the  split  plot  design 
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with  one  between-subjects  factor  and  one  within-subjects 
factor.  Keselman  found  that  some  procedures  including 
Shaffer's  modified  sequentially  rejective  Bonferroni 
procedures  using  the  e -adjusted  F  or  Hotelling's  as  omnibus 
t^sts  became  conservative  when  e  =  .40.  Welsch's  step-up 
procedure  did  not  become  conservative  as  non-sphericity 
increased. 

Table  5  shows  that  when  the  omnibus  null  hypothesis  is 
completely  true,  the  T  x  e  interaction  accounted  for  1.3%  of 
the  total  variance.  Shown  in  Table  27  are  mean  values  of  r 
for  interpreting  the  T  x  e  interaction.  These  results 
indicate  that  when  non-sphericity  increases,  r  decreases 
slightly  for  Bonferroni  procedures  and  Shaffer's  procedures. 
Type  I  error  rates  for  W  and  W,,^.  increase  slightly  when  non- 
sphericity  increases.  The  relatively  small  effect  of 
violating  the  sphericity  assumption  might  be  due  to  the  levels 
of  the  non-sphericity  included  in  this  study  (e  =  .75  and 
.96).     Other  studies  have  included  smaller  values  of  e. 

Effect  of  Nonnormality 

Keselman,  Keselman,  and  Shaffer  (1991)  studied  four 
simultaneous  multiple  comparison  procedures  in  the  split  plot 
design  with  one  between-subjects  factor  and  one  within- 
subjects  factor.  They  found  that  the  Type  I  error  rates  for 
the  Bonferroni  procedure  applied  to  data  sampled  from  a  chi- 
square  distribution  with  three  degrees  of  freedom  are  not 
consistently  larger  or  smaller  than  when  the  Bonferroni 


Table  27 


Mean  Tvoe  I 

Error  Rates  as  a 

Function  of 

Test  and 

Sphericity: 

Comoletelv 

True  Null 

Hypothesis 

and  a  =  .05 

Test 

c  =  .96 

e  =  .75 

B 

0.036 

0.035 

Brt 

0.032 

0.031 

B.DT 

0.047 

0.045 

w 

0.053 

0.057 

Wrt 

0.047 

0.051 

S(e) 

0.035 

0.031 

S(Ort 

0.033 

0.031 

S(e)DT  • 

0.027 

0.027 

S(C)  ' 

0.036 

0.033 

S(C)rt 

0.034 

0.032 

S(C)oT 

0.024 

0.025 

Note.     See  Table  4  for  abbreviations. 
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procedure  is  applied  to  normal  data.  Keselman  (1994) 
investigated  28  stepwise  and  simultaneous  multiple  comparison 
procedures  in  the  split  plot  design  with  one  between-subjects 
factor  and  one  within-subjects  factor.  Keselman  reported  that 
the  Type  I  error  rates  of  the  data  sampled  from  chi-square 
distribution  with  three  degrees  of  freedom  are  quite  similar 
to  those  of  the  data  sampled  from  normal  distribution  for 
Welsch.  The  conservative  nature  of  Shaffer's  modified 
sequentially  rejective  Bonferroni  procedures  was  more  evident 
when  the  data  were  nonnormal. 

Table  5  shows  that  when  the  omnibus  null  hypothesis  is 
completely  true  the  T  x  D  interaction  accounted  for  2.3%  of 
the  total  variance.  Shown  in  Table  28  are  mean  values  of  z 
for  interpreting  T  x  D  interaction.  Results  in  the  table 
indicate  that  Type  I  error  rates  are  smaller  when  the  data  are 
long-tailed.  The  explanation  for  the  disagreement  between  the 
results  in  this  study  and  the  results  in  previous  research  is 
most  likely  that  this  study  included  more  severely  long-tailed 
distribution  than  did  the  previous  studies.  The  chi-square 
distribution  with  three  degrees  of  freedom  has  larger  kurtosis 
than  the  distribution  with  h  =  .109,  but  it  has  much  smaller 
kurtosis  than  the  distribution  with  h  =  .35.  In  addition  the 
distributions  included  in  this  study  were  symmetric;  the 
nonnormal  distributions  in  the  other  studies  were  asymmetric. 


Table  28 


Mean  Type  I  Error  Rates  as  a  Function  of 
Test  and  Distribution:  Completely  True  Null 
Hypothesis  and  a  =  .05 


Test 

h=0 

h=. 109 

h=.35 

B 

0 .  042 

0  .038 

0  .  027 

0  .035 

0  .  033 

0.028 

Bdt 

0 . 050 

0 . 047 

0  .  040 

W 

0  .060 

0  .  057 

0.047 

0.053 

0.050 

0.044 

S(e) 

0.039 

0.036 

0.023 

0.034 

0.033 

0.029 

•  0.030 

0.028 

0.023 

S(C) 

0.041 

0.038 

0.025 

S(C)«T 

0.035 

0.034 

0.029 

S(C)pT 

0.028 

0.026 

0.021 

Note.     See  Table  4  for  abbreviations. 


Effect  of  Trimmed  Means 

Yuen  (1974)  reported  that  in  the  case  of  unequal  sample 
sizes  and  variances  the  power  of  the  trimmed  Welch  test  is 
superior  to  the  power  of  Welch's  test  for  long-tailed 
distributions.  Wilcox  (1992)  investigated  the  effect  of 
nonnormality  on  Type  I  error  rate  and  power  when  comparing  two 
independent  groups.  Wilcox  showed  that  the  power  of  Welch's 
procedure  drops  dramatically  as  the  amount  of  contamination 
gets  large  and  Yuen's  trimmed  Welch  test  has  more  power  than 
Welch's  test.  The  results  of  this  study  are  consistent  with 
the  findings  of  the  previous  research.  Table  15  contains 
means  of  any-pair  power  as  a  function  of  test  and 
distribution.  Results  in  the  table  indicate  that  the 
procedures  using  trimmed  data  have  more  any-pair  power  than 
their  counterparts  as  the  distribution  has  long  tails. 

Power  Comparisons 

Keselman  (1994)  investigated  28  pairwise  multiple 
comparison  procedures  in  the  split  plot  design  with  one 
between-subjects  factor  and  one  within-subjects  factor. 
Keselman  concluded  that  the  Welsch  step-up  test  tends  to  have 
the  best  any-pair  and  average  per-pair  power  while  the 
Bonferroni  procedure  has  the  least  per-pair  power.  The 
conclusions  of  this  study  is  consistent  with  the  findings  of 
the  previous  research.  The  results  indicate  that  W  and  W^^ 
have  the  highest  any-pair  and  average  per-pair  power  and  B, 
B^j,    and  Bp,,   have  the  lowest  per-pair  power. 
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Effect  of  c  on  Type  I  Error  Rates  for  Shaffer's  Test 
Algina  and  Oshima  (in  press)  investigated  the  effect  of 
heterogeneous  v-c  matrices  on  the  Type  I  error  rates  of 
adjusted  F  tests  using  unweighted  means  in  a  split  plot 
design.  When  the  sample  sizes  are  equal,  the  e-adjusted  test 
provides  good  control  of  Type  I  error  rate  even  when  the  v-c 
matrices  are  heterogeneous.  However,  when  the  sample  sizes 
are  unequal  and  v-c  matrices  are  heterogeneous,  the  e-adjusted 
test  provides  poor  control  of  the  Type  I  error  rate.  Algina 
and  Oshima  concluded  that  when  the  sample  sizes  are  not  equal, 
the  CIGA  test  control  the  Type  I  error  rate  much  better  than 
does  the  e-adjusted  test. 

The  results  of  this  study  are  consistent  with  the 
previous  research.  Results  in  Table  7  indicate  that  when  the 
sample  sizes  are  equal,  Shaffer's  procedures  following  e- 
adjusted  omnibus  F  tests  have  good  control  of  Type  I  error 
rates  even  when  the  v-c  matrices  are  heterogeneous.  However, 
when  the  sample  sizes  are  unequal  and  v-c  matrices  are 
heterogeneous,  Shaffer's  procedures  following  e-adjusted  F 
test  provide  poor  control  of  Type  I  error  rates.  The  poor 
performance  of  Shaffer's  tests  is  due  to  the  impact  of  e  on 
the  Type  I  error  rate  of  e-adjusted  F  test.  Shaffer's 
procedures  following  CIGA  test  have  good  control  of  Type  I 
error  rates  under  heteroscedasticity  when  the  sample  sizes  are 
equal  or  unequal . 
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Conclusions 

Four  hundred  and  thirty-two  simulated  conditions  were 
investigated  in  a  complete  factorial  experiment.  The 
generalization  of  the  results  is  limited  by  the  range  of 
values  for  the  distribution  of  the  generated  data,  sphericity 
of  the  common  v-c  matrix,  degree  of  the  heteroscedasticity  of 
the  v-c  matrices,  total  sample  size,  sample  size  arrangement, 
the  configuration  of  the  nonnull  means.  With  these 
limitations  the  following  conclusions  may  be  set  forth: 

Conclusion  1.  The  W  and  W^^  tests  have  Type  I  error 
rates  closest  to  the  nominal  significance  level.  However,  the 
W  test  appears  to  have  the  strongest  liberal  tendency. 
Especially  under  conditions  where  the  data  are  normal  and  e  is 
small,  those  concerned  about  liberal  tests  might  avoid  W  and 
use  either  Wp,.   or  Bp^. 

Conclusion  2.  The  W  test  has  the  highest  power  when  the 
distribution  is  normal  or  slightly  kurtotic.  However,  W  has 
a  liberal  tendency  in  normal  distribution  as  mentioned  in 
Conclusion  1.  The  Wp^,  B,  B^^,  and  S(C)  tests  also  perform 
well  in  terms  of  power  when  the  distribution  is  slightly 
kurtotic.  In  the  very  long-tailed  distribution,  Wp^  has  the 
highest  powers.  "  ' 

Conclusion  3.  The  tests  using  trimmed  data  have  about  10 
to  25%  less  any-pair  power  and  about  25  to  40%  less  average 
per-pair  power  than  their  non-trimmed  counterparts  when  the 
distribution  is  normal. 
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Conclusion  4.  The  W  and  W^^  tests  have  high  power  under 
negative  pairing  of  v-c  matrices  and  sample  sizes;  Shaffer's 
procedures  following  e -adjusted  F  test  have  low  power  under 
positive  pairing.  Under  negative  pairing  and  large  v-c 
heteroscedasticity  the  power  of  the  Shaffer  tests  rival  that 
of  W.  However,  under  the  cited  condition  the  Shaffer  tests 
tend  to  be  liberal. 

Conclusion  5.  The  W  and  W^^  tests  have  the  highest 
average  per-pair  power.  The  Bonferroni  procedures  have  low 
average  per-pair  power.  •  -  ■ 

Conclusion  6.  The  impact  of  mean  configuration  on  any- 
pair  power  and  average  per-pair  power  is  smaller  for  the 
Shaffer  tests  than  it  is  for  the  Welsch  and  Bonferroni  tests. 

Suggestions  for  Future  Research 

The  generalizability  of  the  study  is  limited  by  (a)  the 
number  of  distributions  considered,  (b)  the  forms  of 
dispersion  matrices  considered,  (c)  the  variation  in  sample 
size  arrangement,  and  (d)  the  variation  in  the  power  of  the 
omnibus  tests.  This  limitations  suggest  several  lines  for 
additional  research. 

First,  the  inclusion  of  the  normal  distribution  (h  =  0) 
and  the  distributions  with  h  =  .109  and  h  =  .35  gave 
representation  to  different  levels  of  kurtosis.  Further 
research  needs  to  be  conducted  to  see  how  the  tests  perform 
and  how  factors  included  in  the  study  are  affected  when  (a) 


data  are  generated  from  skewed  distributions  and  (b)  data  are 
generated  from  skewed  and  long-tailed  distributions.  In 
addition  short-tailed  distributions  should  be  investigated 
since  the  Type  I  error  rate  for  the  Welsch  test  tended  to 
increase  as  kurtosis  declined  and  to  be  above  a  when  the  data 
were  normal . 

Second,  additional  forms  of  dispersion  matrices  should  be 
considered.  For  example,  the  autocorrelation  (AR)  models 
might  be  considered.  First-  and  second-order  AR  models  are 
attractive  for  the  split-plot  or  repeated  measurements 
designs. 

Third,  additional  permutations  of  the  v-c  matrices 
included  in  the  study  should  be  investigated,  since  the 
particular  permutations  included  in  this  study  affected  power 
through  the  ratio  in  equation  (52). 

Fourth,  only  cases  of  complete  data  were  considered. 
Since  incomplete  data  (i.e.,  data  for  some  conditions  are 
missing)  are  common  in  the  application  of  the  split-plot  or 
repeated  measurements  designs,  it  would  be  useful  to  examine 
the  tests  when  data  are  incomplete.  Also,  approaches  that 
does  not  rely  on  complete  data  may  be  considered.  When  some 
subjects  have  incomplete  data,  functions  can  be  defined  to 
take  the  missing  data  into  account  (e.g.,  restriction  to  those 
subjects  with  responses  to  both  conditions  for  a  pairwise 
difference)  or  functions  that  have  meaningful  definitions  for 
all  subjects  (e.g.,  the  median)  can  be  considered. 


Fifth,  in  all  conditions  the  power  for  the  omnibus  test 
was  approximately   .50.      However,    since  all-pairs  power  was 
very  low  for  all  tests,   it  would  be  useful  to  examine  the 
tests  using  the  noncentrality  parameter  to  produce  more  power 
(e.g.,  omnibus  power  of  .90). 

Finally,  since  nonnormality  is  common,  investigating 
approaches  that  do  not  rely  on  normality  would  be  useful. 
Nonparametric  methods  can  be  used  when  the  normal  distribution 
assumption  is  not  realistic.  Symmetry  and  independence  of  the 
distribution  are  sufficient  to  support  the  sign  test  for 
pairwise  comparisons,  and  if  the  pairwise  comparison  can  also 
be  ranked,  the  Wilcoxon  signed  rank  test  is  applicable.  The 
bootstrap  provides  an  alternative  approach  to  assessing  the 
distributional  properties  of  estimators  without  relying  on 
normality  assumption.  The  bootstrap  provides  an  estimate  of 
the  sampling  distribution  of  an  estimator  based  only  on  the 
estimation  procedure  and  the  sample  data  in  hand  without 
appealing  to  distributional  assumption.  The  major  limitation 
of  the  bootstrap  in  this  application  may  be  its  heavy  use  of 
computer  time. 


APPENDIX 

ESTIMATED  TYPE   I  ERROR  RATES  AND  POWER 

Table  29  includes  estimated  Type  I  error  rates  for  the 
triirmed  difference  score  Bonferroni  (B^^),  Welsch  (W),  raw 
score  trimmed  Welsch  (W^^),  and  Shaffer'  procedure  following 
the  CIGA  test  at  nominal  Type  I  error  rate  .05  for  differing 
levels  of  total  sample  size,  sample  size  arrangement, 
distribution,  sphericity,  and  heteroscedasticity  of  v-c 
matrices . 

Table  30  and  Table  31  include  estimated  any-pair  and 
average  per-pair  power  for  the  trimmed  difference  score 
Bonferroni  (B^j) ,  Welsch  (W),  raw  score  trimmed  Welsch  (Wp^), 
and  Shaffer'  procedure  following  the  CIGA  test  at  nominal  Type 
I  error  rate  .05  for  differing  levels  of  configuration  of 
means,  total  sample  size,  sample  size  arrangement, 
distribution,  sphericity,  and  heteroscedasticity  of  v-c 
matrices,  respectively.  Only  results  for  four  most  effective 
procedures  are  reported.  Only  any-pair  power  and  average  per- 
pair  power  are  reported  since  all-pairs  power  are  very  low  for 
all  procedures. 

The  levels  of  the  factors  are  the  following:  (a)  for 
n^/nj,  "A"  denotes  positive  pairing  of  v-c  matrices  and  sample 
sizes,     "B"    denotes    equal    sample    sizes,    and    "C"  denotes 
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negative  pairing  of  v-c  matrices  and  sample  sizes;  (b)  for  D, 
"1"  denotes  the  sampled  distribution  was  normal,  "2"  denotes 
the  sampled  distribution  was  h  =  .109,  and  "3"  denotes  the 
sampled  distribution  was  h  =  .35;  (c)  for  e,  "1"  denotes  e  - 
.96  whereas  "2"  denotes  e  =  .75;  and  (d)  for  S^rSj,  "1" 
denotes  E^tSj  =1:1,  "2"  denotes  S^rSj  =1:2,  and  "3"  denotes 
S^:S2  =  1:5.  Reported  Type  I  error  rates,  any-pair  power,  and 
average  per-pair  power  are  of  the  form  10000  times  estimated 
rates . 


Table  29 

Estimated  Type  I  Error  Rates:  Completely  True  Null 
Hypothesis  and  a  =  .05 


N 

Hi /rip 

D 

e 

Sji  •  So 
1  C. 

W 

K  1 

S(C) 

40 

A 

1 

1 

1 

0612 

0664 

0552 

0442 

40 

A 

1 

1 

2 

0456 

0560 

0464 

0452 

40 

A 

1 

1 

3 

0478 

0542 

0428 

0438 

40 

A 

1 

2 

1 

0580 

0692 

0596 

0376 

40 

A 

1 

2 

2 

0478 

0656 

0552 

0384 

40 

A 

1 

2 

3 

0480 

0600 

0528 

0376 

40 

A 

2 

1 

1 

0544 

0646 

0502 

0420 

40 

A 

2 

1 

2 

0462 

0496 

0440 

0374 

40 

A 

2 

1 

3 

0394 

0488 

0490 

0394 

40 

A 

2 

2 

1 

0476 

0684 

0540 

0358 

40 

A 

2 

2 

2 

0400 

0544 

0444 

0330 

40 

A 

2 

2 

3 

0390 

0518 

0472 

0302 

40 

A 

3 

1 

1 

0406 

0440 

0436 

0250 

40 

A 

3 

1 

2 

0390 

0388 

0372 

0260 

40 

A 

3 

1 

3 

0408 

0450 

0392 

0268 

40 

A 

3 

2 

1 

0360 

0446 

0450 

0226 

40 

A 

3 

2 

2 

0384 

0544 

0472 

0284 

40 

A 

3 

2 

3 

0330 

0456 

0370 

0220 

40 

B 

1 

1 

1 

0430 

0528 

0418 

0448 

40 

B 

1 

1 

2 

0462 

0524 

0452 

0402 
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Table  29 — continued. 


N  n^/nj 


D 


B 


DT 


w 


w 


RT 


S(C) 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 


1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


0528 
0476 
0420 
0496 
0488 
0496 
0504 
0388 
0426 
0490 
0380 
0356 
0394 
0408 
0372 
0374 
0506 
0616 
0624 
0506 
0600 


0600 
0652 
0576 
0582 
0520 
0544 
0580 
0574 
0542 
0678 
0442 
0430 
0460 
0526 
0516 
0422 
0580 
0682 
0656 
0636 
0680 


0528 
0562 
0472 
0542 
0458 
0430 
0496 
0456 
0452 
0496 
0360 
0396 
0414 
0406 
0406 
0436 
0490 
0624 
0630 
0546 
0576 


0442 
0416 
0356 
0346 
0418 
0390 
0412 
0314 
0354 
0396 
0272 
0234 
0246 
0252 
0290 
0244 
0392 
0410 
0378 
0404 
0354 
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Table  29 — continued. 


N 

n^/nj 

D 

e 

^1  *  ^2 

W 

S(C) 

40 

C 

1 

2 

3 

0656 

0730 

0664 

0428 

40 

C 

2 

1 

1 

0476 

0512 

0524 

0352 

40 

c 

2 

1 

2 

0560 

0586 

0542 

0354 

40 

c 

2 

1 

3 

0574 

0636 

0576 

0334 

40 

c 

2 

2 

1 

0490 

0638 

0530 

0384 

40 

c 

2 

2 

2 

0554 

0638 

0590 

0382 

40 

c 

2 

2 

3 

0634 

0744 

0594 

0400 

40 

c 

3 

1 

1 

0440 

0456 

0460 

0230 

40 

c 

3 

1 

2 

0420 

0478 

0432 

0216 

40 

c 

3 

1 

3 

0454 

0466 

0420 

0206 

40 

c 

3 

2 

1 

0428 

0496 

0478 

0236 

40 

c 

3 

2 

2 

0442 

0470 

0452 

0214 

40 

c 

3 

2 

3 

0502 

0528 

0558 

0268 

60 

A 

1 

1 

0528 

0548 

0502 

0404 

60 

A 

1 

2 

0478 

0560 

0482 

0466 

60 

A 

1 

3 

0500 

0542 

0484 

0464 

60 

A 

2 

1 

0492 

0700 

0576 

0418 

60 

A 

2 

2 

0496 

0622 

0554 

0384 

60 

A 

2 

3 

0446 

0630 

0564 

0384 

60 

A 

2 

1 

1 

0452 

0588 

0474 

0434 

60 

A 

2 

1 

2 

0488 

0550 

0508 

0414 

118 

Table  29--continued . 


N      n-,/n2     D        e  S^iS^ 


B 


DT 


w 


w, 


RT 


S(C) 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 


2 

2 

2 

2 

3 

3 

3 

3 

3 

3 

1 

1 

1 

1 

1 

1 

2 

2 

2 

2 

2 


1 

2 

2 

2 

1 

1 

1 

2 

2 

2 

1 

1 

1 

2 

2 

2 

1 

1 

1 

2 

2 


3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 


0446 

0430 

0448 

0418 

0422 

0388 

0366 

0402 

0400 

0376 

0466 

0440 

0494 

0460 

0428 

0464 

0462 

0498 

0460 

0412 

0422 


0542 
0528 
0594 
0534 
0456 
0448 
0458 
0546 
0484 
0498 
0538 
0546 
0502 
0578 
0606 
0594 
0546 
0578 
0482 
0562 
0582 


0470 
0502 
0526 
0506 
0494 
0494 
0422 
0476 
0464 
0470 
0478 
0484 
0494 
0542 
0508 
0572 
0462 
0486 
0422 
0484 
0518 


0448 
0312 
0396 
0348 
0236 
0294 
0288 
0278 
0266 
0282 
0488 
0472 
0404 
0324 
0384 
0348 
0458 
0462 
0372 
0322 
0346 
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Table  29--continued. 


D 


B 


DT 


w 


w 


RT 


S(C) 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


B 
B 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 


2 
3 
3 
3 
3 
3 
3 


2 
2 
2 
2 
2 
2 
3 
3 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


0414 
0390 
0428 
0402 
0388 
0428 
0392 
0534 
0540 
0498 
0448 
0520 
0508 
0450 
0514 
0530 
0438 
0478 
0514 
0372 
0416 


0530 
0430 
0468 
0472 
0506 
0468 
0448 
0586 
0616 
0594 
0608 
0592 
0614 
0536 
0554 
0596 
0560 
0622 
0580 
0422 
0434 


0520 
0416 
0416 
0418 
0472 
0450 
0472 
0492 
0554 
0514 
0526 
0584 
0582 
0480 
0518 
0526 
0536 
0590 
0510 
0444 
0424 


0304 
0266 
0308 
0314 
0290 
0258 
0242 
0458 
0458 
0432 
0370 
0368 
0348 
0398 
0396 
0392 
0364 
0372 
0326 
0246 
0232 
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Table  29 — continued . 


N 

n^/nj 

D 

e 

W 

Wrt 

S(C) 

60 

C 

3 

1 

3 

0420 

0508 

0438 

0252 

60 

C 

3 

2 

1 

0388 

0472 

0454 

0244 

60 

c 

3 

2 

2 

0372 

0424 

0482 

0188 

60 

c 

3 

2 

3 

0354 

0452 

0472 

0248 

Table  30 

Estimated  Any-Pair  Power:  a  =  .05 
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CM    N     n^/nj   D      e  S^rSj 


B 


DT 


w 


w, 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 

40 


A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

A 

B 

B 


1 

1 

1 

1 

1 

1 

2 

2 

2 

2 

2 

2 

3 

3 

3 

3 

3 

3 

1 

1 


1 

1 

1 

2 

2 

2 

1 

1 

1 

2 

2 

2 

1 

1 

1 

2 

2 

2 

1 

1 


1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 


3232 

3794 

4430 

4314 

5072 

5938 

2612 

3024 

3590 

3728 

4276 

5164 

1758 

1894 

2282 

2368 

2754 

3238 

3272 

3216 


4124 

4828 

5534 

5958 

6600 

7490 

3020 

3540 

4192 

4504 

5270 

6052 

1520 

1588 

1922 

2216 

2458 

2790 

4282 

4272 


3186 

3844 

4440 

4574 

5164 

6024 

2756 

3144 

3848 

3876 

4444 

5344 

1958 

2254 

2670 

2760 

3106 

3756 

3166 

3140 


4032 

4924 

5672 

3850 

4502 

5230 

2878 

3456 

4150 

2674 

3162 

3744 

1114 

1144 

1446 

1034 

1108 

1220 

4322 

4340 
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Table  30--continued, 


CM    N     n^/n2   D      e  2^:E2 


B 


DT 


w 


w 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 


1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


3260 
4384 
4402 
4428 
2750 
2734 
2718 
3682 
3694 
3732 
1634 
1652 
1702 
2430 
2386 
2438 
3286 
2878 
2492 
4332 
3920 


4228 
6136 
6030 
6060 
3178 
3202 
3272 
4608 
4724 
4754 
1410 
1486 
1424 
2054 
2126 
2204 
4262 
3686 
3390 
5810 
5432 


3154 
4344 
4380 
4358 
2688 
2736 
2848 
3782 
3856 
3808 
1898 
1870 
1910 
2668 
2754 
2736 
3316 
2840 
2524 
4294 
3890 


4238 
4004 
3802 
3884 
3070 
3146 
3040 
2646 
2668 
2678 
1024 
1146 
1030 
936 
926 
946 
4142 
3608 
3028 
3776 
3330 


Table  30--continued, 
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CM    N     n^/n2   D      e  2^:S2 


B 


DT 


w 


w, 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
60 
60 
60 
60 
60 
60 
60 
60 


C 
C 
C 
C 
C 
C 
C 

c 
c 
c 
c 
c 
c 

A 
A 
A 
A 
A 
A 
A 
A 


1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


3374 
2688 
2400 
2080 
3606 
3108 
2906 
1652 
1590 
1310 
2376 
2060 
1772 
3390 
3960 
4692 
4526 
5340 
6212 
2740 
3066 


4782 
3194 
2850 
2562 
4686 
4088 
3754 
1566 
1484 
1366 
2188 
1984 
1952 
4162 
4912 
5Mw6 
5968 
6796 
7774 
3146 
3518 


3386 
2744 
2424 
2128 
3932 
3350 
2846 
1892 
1778 
1518 
2648 
2320 
2060 
3192 
3902 
4588 
4580 
5382 
6332 
2838 
3272 


2822 
2916 
2472 
2176 
2762 
2294 
2026 
1098 
978 
870 
940 
884 
862 
4192 
5078 
5870 
3712 
4456 
5430 
3072 
3508 
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Table  30 — continued , 


CM    N    n^/nj    D      e  2^:22 


B 


DT 


w 


w, 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 


2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


3866 
3888 
4504 
5206 
1884 
2042 
2402 
2418 
2834 
3434 
3372 
3242 
3312 
4756 
4740 
4692 
2878 
2780 
2774 
3946 
3780 


4170 
4688 
5248 
5968 
1452 
1586 
1696 
1982 
2280 
2526 
4364 
4088 
4212 
6268 
6286 
6128 
3154 
3120 
3222 
4722 
4604 


3966 
4002 
4624 
5436 
2070 
2304 
2720 
2754 
3232 
3890 
3284 
3172 
3282 
4738 
4740 
4604 
2962 
2828 
2814 
4082 
3968 


4284 
2754 
3100 
3754 
1126 
1264 
1366 
932 
1070 
1212 
4530 
4272 
4298 
4064 
3980 
3928 
3266 
3110 
3116 
2660 
2630 
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Table  30 — continued . 


CM    N     n^/nj    D  £2^:22 


B 


DT 


w 


w, 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


B 
B 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 


2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


3870 
1824 
1798 
1898 
2590 
2530 
2486 
3344 
3008 
2682 
4610 
4020 
3538 
2746 
2556 
2138 
3940 
3398 
2966 
1754 
1464 


4654 
1332 
1366 
1520 
2050 
1978 
2000 
4186 
3826 
3412 
6102 
5466 
4892 
3168 
2908 
2568 
4728 
4320 
3784 
1450 
1348 


4014 
2072 
1978 
2090 
2874 
2862 
2838 
3394 
2966 
2646 
4652 
4104 
3620 
2886 
2632 
2304 
3978 
3636 
3038 
1996 
1772 


2634 
1040 
1054 
1220 
936 
934 
924 
4256 
3764 
3240 
3956 
3378 
2874 
3068 
2754 
2216 
2652 
2492 
2050 
1122 
1004 
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Table  30--continued, 


CM     N     n^/nj    D       e  S^tEj 


B 


DT 


w 


w, 


RT 


S(C) 


1 
1 
1 
1 

2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


60 
60 
60 
60 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


C 

c 
c 
c 

A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 


3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1338 
2346 
2150 
1838 
3724 
4332 
5288 
2760 
3052 
3582 
3110 
3446 
4200 
2286 
2516 
2852 
1880 
2072 
2652 
1570 
1616 


1312 
2058 
1914 
1812 
4912 
5526 
6482 
3626 
4214 
4824 
3690 
3970 
4924 
2782 
3136 
3490 
1742 
1762 
2152 
1514 
1506 


1586 
2820 
2492 
2138 
3702 
4200 
5176 
2904 
3362 
3840 
3194 
3580 
4460 
2544 
2948 
3342 
2246 
2516 
3168 
1962 
2152 


920 
960 
814 
768 
4446 
5236 
6286 
3396 
3960 
4660 
3168 
3568 
4494 
2450 
2894 
3196 
1114 
1210 
1396 
1088 
1094 


Table  30--continued . 
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CM    N     n^/nj   D      e  S^:E2 


B 


DT 


w 


w, 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


A 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 


3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1792 
3866 
3878 
3730 
2640 
2652 
2804 
3152 
3152 
3098 
2144 
2176 
2214 
1912 
1912 
1940 
1456 
1498 
1504 
3910 
3268 


1604 
4928 
4952 
4942 
3714 
3672 
3790 
3662 
3794 
3704 
2770 
2750 
2782 
1592 
1622 
1698 
1326 
1346 
1396 
5050 
4166 


2398 
3610 
3558 
3552 
2790 
2782 
2870 
3122 
3204 
3286 
2490 
2492 
2502 
2242 
2302 
2266 
1810 
1742 
1874 
3764 
3090 


1238 
4638 
4612 
4498 
3476 
3446 
3504 
3332 
3342 
3130 
2582 
2480 
2470 
1054 
1026 
1066 

990 
1008 

982 
4724 
3756 


Table  30 — continued, 


CM    N    n^/nj   D      e  2^:22 


B 


DT 


w 


w, 


RT 


S(C) 
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2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
60 
60 
60 
60 
60 


C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
A 
A 
A 
A 
A 


1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


2928 
2670 
2530 
2220 
3118 
2748 
2408 
2182 
2034 
1842 
1950 
1702 
1498 
1464 
1374 
1262 
3944 
4670 
5388 
2708 
3098 


3756 
3638 
3274 
3092 
3818 
3322 
3016 
2848 
2514 
2394 
1724 
1696 
1512 
1396 
1352 
1308 
4944 
5672 
6428 
3572 
4182 


2688 
2928 
2608 
2388 
3260 
2954 
2492 
2510 
2314 
2024 
2254 
2056 
1820 
1896 
1776 
1564 
3778 
4476 
5264 
2904 
3312 


3146 
3380 
2932 
2596 
3132 
2674 
2284 
2508 
2068 
1960 
1086 
1018 
830 
996 
934 
824 
4602 
5432 
6216 
3398 
3958 


Table  30 — continued . 
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CM    N    n<,/n2   D      e  H^rSj 


B 


DT 


w 


w, 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
B 
B 
B 
B 
B 
B 
B 
B 


1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


3788 
3212 
3852 
4476 
2332 
2534 
3046 
1946 
2212 
2760 
1474 
1672 
1968 
3982 
4082 
3856 
2782 
2674 
2784 
3104 
3222 


4954 
3632 
4252 
4782 
2880 
3100 
3598 
1672 
1730 
2094 
1370 
1448 
1584 
5018 
5040 
4884 
3808 
3694 
3672 
3522 
3654 


3986 
3238 
3960 
4524 
2728 
3020 
3466 
2310 
2696 
3264 
2052 
2218 
2596 
3818 
3850 
3672 
3020 
2970 
2902 
3252 
3314 


4806 
3192 
3916 
4468 
2596 
2918 
3350 
1118 
1208 
1432 
970 
1064 
1280 
4838 
4756 
4592 
3602 
3562 
3448 
3198 
3324 


Table  30 — continued, 
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CM     N     n-,/n2    D       8  S^rSj 


B 


DT 


W, 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 


2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


3254 
2120 
2230 
2290 
2070 
2096 
2146 
1544 
1480 
1454 
3918 
3530 
3084 
2700 
2498 
2176 
3130 
2776 
2550 
2306 
2060 


3728 
2658 
2652 
2784 
1492 
1470 
1622 
1284 
1234 
1286 
4918 
4432 
3964 
3584 
3302 
2938 
3602 
3274 
3034 
2770 
2578 


3338 
2562 
2620 
2644 
2380 
2408 
2412 
1912 
2006 
1834 
3760 
3364 
2936 
2824 
2672 
2364 
3204 
2904 
2640 
2690 
2276 


3318 
2464 
2470 
2522 
1044 
1034 
1092 
984 
932 
994 
4614 
4002 
3464 
3376 
3060 
2682 
3218 
2802 
2550 
2518 
2246 


Table  30 — continued. 
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CM    N     n^/nj   D      e  E^rSj 


B 


DT 


w 


w, 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


60 
60 
60 
60 
60 
60 
60 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


C 
C 
C 
C 
C 
C 
C 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 


2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 


2 

1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1840 
1950 
1730 
1610 
1582 
1274 
1300 
3572 
4126 
4954 
3038 
3366 
3976 
2970 
3330 
4020 
2606 
2870 
3348 
1848 
2024 


2362 
1654 
1510 
1470 
1374 
1248 
1266 
4564 
5266 
6038 
4168 
4678 
5470 
3622 
3938 
4600 
3166 
3628 
4190 
1644 
1838 


2100 
2390 
2042 
1776 
2098 
1750 
1610 
3560 
4060 
4818 
3340 
3682 
4298 
3132 
3462 
4202 
2880 
3330 
3824 
2158 
2410 


1990 
1152 
942 
876 
1016 
926 
810 
4344 
5006 
6006 
3442 
3994 
4798 
3102 
3618 
4352 
2528 
2974 
3452 
1058 
1270 
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Table  30 — continued . 


CM    N    n^/nj   D      e  2-,:S2 


B 


DT 


w 


w, 


RT 


S(C) 


3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


A 

A 

A 

A 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 


3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


2444 
1700 
1758 
2028 
3540 
3610 
3572 
3056 
2866 
2936 
2938 
2912 
3048 
2542 
2418 
2526 
1776 
1732 
1978 
1568 
1540 


2138 
1650 
1666 
1900 
4604 
4578 
4616 
4160 
4194 
3990 
3532 
3442 
3534 
3268 
3124 
3230 
1480 
1562 
1742 
1494 
1464 


2890 
2084 
2340 
2716 
3378 
3506 
3428 
3150 
3026 
3056 
3144 
2918 
3016 
2880 
2788 
2778 
2114 
2052 
2276 
2082 
2088 


1514 
1050 
1094 
1244 
4528 
4464 
4402 
3570 
3482 
3300 
3314 
3250 
3150 
2632 
2510 
2568 
1070 
1094 
1168 
990 
966 


1 
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Table  30--continued . 


CM 

N 

D 

e 

W 

S(C) 

3 

40 

B 

3 

2 

3 

1584 

1580 

2010 

986 

3 

40 

C 

1 

1 

1 

3666 

4684 

3550 

4348 

3 

40 

C 

1 

1 

2 

3292 

4072 

3064 

3756 

3 

40 

C 

1 

1 

3 

2894 

3734 

2816 

3190 

3 

40 

C 

1 

2 

1 

3068 

4214 

3386 

3512 

3 

40 

C 

1 

2 

2 

2802 

3782 

2956 

3110 

3 

40 

C 

1 

2 

3 

2506 

3412 

2630 

2698 

3 

40 

C 

2 

1 

1 

2974 

3492 

3094 

3124 

3 

40 

C 

2 

1 

2 

2716 

3266 

2802 

2736 

3 

40 

C 

2 

1 

3 

2298 

2824 

2404 

2282 

3 

40 

C 

2 

2 

1 

2654 

3260 

2872 

2560 

3 

40 

C 

2 

2 

2 

2336 

2924 

2510 

2104 

3 

40 

C 

2 

2 

3 

2110 

2670 

2322 

1910 

3 

40 

C 

3 

1 

1 

1848 

1688 

2130 

1068 

3 

40 

C 

3 

1 

2 

1638 

1564 

1956 

958 

3 

40 

C 

3 

1 

3 

1458 

1524 

1776 

902 

3 

40 

C 

3 

2 

1 

1634 

1612 

2038 

1030 

3 

40 

C 

3 

2 

2 

1486 

1522 

1924 

904 

3 

40 

C 

3 

2 

3 

1458 

1526 

1726 

864 

3 

60 

A 

1 

1 

1 

3926 

4832 

3722 

4698 

3 

60 

A 

1 

1 

2 

4184 

5204 

4002 

5108 

•i 


Table  30 — continued. 


CM    N    n^/nj   D      e  S^rSj 


B 


DT 


w 


w 


RT 


S(C) 


3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
B 
B 
B 
B 
B 


1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


5026 
3012 
3538 
4008 
2916 
3458 
4110 
2636 
2804 
3498 
1758 
2064 
2630 
1770 
1792 
2188 
3692 
3756 
3694 
3080 
3100 


6020 
4026 
4734 
5342 
3398 
3888 
4528 
3172 
3448 
4142 
1494 
1754 
1950 
1486 
1542 
1770 
4502 
4658 
4648 
4232 
4264 


4838 
3266 
3722 
4232 
3124 
3636 
4316 
2970 
3206 
3926 
2084 
2558 
3018 
2254 
2376 
2862 
3418 
3584 
3512 
3216 
3270 


5950 
3344 
4060 
4684 
3142 
3686 
4350 
2548 
2830 
3472 
1056 
1258 
1486 
1022 
1028 
1274 
4442 
4546 
4516 
3534 
3616 


Table  30--continued. 


135 


CM    N     n^/Hj    D       e  E^iSj       Bp^  W  Wp^  S(C) 


3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 

c 
c 
c 
c 
c 
c 


1 

2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


3096 
3024 
2978 
3052 
2528 
2574 
2666 
1988 
1902 
1904 
1672 
1734 
1792 
3742 
3238 
2880 
3000 
2722 
2458 
3078 
2614 


4176 
3446 
3386 
3434 
3152 
3228 
3250 
1510 
1516 
1544 
1426 
1470 
1524 
4612 
4104 
3664 
4166 
3752 
3352 
3464 
3084 


3244 
3178 
3134 
3070 
2884 
2920 
2890 
2236 
2220 
2224 
2254 
2240 
2236 
3602 
3110 
2758 
3256 
3014 
2612 
3262 
2738 


3452 
3256 
3204 
3204 
2516 
2582 
2574 
1128 
1070 
1062 
972 
986 
974 
4482 
3902 
3292 
3444 
3100 
2678 
3144 
2720 


J 
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Table  30 — continued. 


CM 

N 

n^/nj 

D 

e 

W 

Wrt 

S(C) 

3 

60 

C 

2 

1 

3 

2368 

2828 

2540 

2388 

3 

60 

c 

2 

2 

1 

2530 

3124 

2908 

2490 

3 

60 

c 

2 

2 

2 

2206 

2854 

2634 

2234 

3 

60 

c 

2 

2 

3 

1970 

2534 

2288 

1840 

3 

60 

c 

3 

1 

1 

1878 

1598 

2212 

1152 

3 

60 

c 

3 

1 

2 

1696 

1470 

1968 

944 

3 

60 

c 

3 

1 

3 

1486 

1414 

1708 

860 

3 

60 

c 

3 

2 

1 

1672 

1454 

2246 

922 

3 

60 

c 

3 

2 

2 

1512 

1458 

2000 

934 

3 

60 

c 

3 

2 

3 

1360 

1418 

1708 

896 

I 


Table  31 

Estimated  All-Pairs  Power:  a  =  .05 
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CM    N    n^/nj    D      e  S^rSj 


B 


OT 


W 


w, 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
B 
B 


1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 


1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1177 
1460 
1811 
1549 
1920 
2389 
886 
1092 
1387 
1281 
1513 
1920 
559 
620 
769 
745 
893 
1121 
1183 
1166 


2979 
3608 
4229 
5346 
5974 
6951 
2118 
2492 
3068 
3876 
4613 
5416 
979 
1057 
1251 
1802 
2005 
2305 
3099 
3112 


2191 
2692 
3243 
3879 
4470 
5336 
1868 
2152 
2726 
3248 
3775 
4640 
1308 
1508 
1833 
2191 
2525 
3116 
2153 
2151 


1873 
2401 
2947 
2004 
2450 
3004 
1235 
1548 
1986 
1300 
1567 
2003 
430 
463 
580 
438 
483 
538 
2011 
2040 


Table  31 — continued . 
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CM    N    n-i/nj   D      £  2^:S2 


B 


DT 


w 


w 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 


1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1188 
1592 
1587 
1572 
969 
952 
964 
1282 
1247 
1256 
531 
547 
549 
758 
758 
768 
1167 
972 
832 
1500 
1331 


3077 
5481 
5416 
5401 
2275 
2310 
2351 
3963 
4093 
4124 
935 
954 
940 
1655 
1751 
1793 
3091 
2599 
2351 
5131 
4727 


2175 
3746 
3755 
3706 
1880 
1894 
1978 
3160 
3221 
3165 
1298 
1251 
1288 
2156 
2189 
2213 
2291 
1931 
1653 
3602 
3183 


1997 
2123 
2029 
2034 
1385 
1379 
1348 
1312 
1305 
1306 
404 
431 
411 
383 
403 
394 
1912 
1596 
1294 
1916 
1610 
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Table  31 — continued . 


CM 

N 

D 

e 

W 

S(C) 

1 

40 

C 

1 

2 

3 

1102 

4089 

2742 

1325 

1 

40 

C 

2 

1 

1 

920 

2217 

1873 

1263 

1 

40 

C 

2 

1 

2 

786 

1931 

1601 

1036 

1 

40 

C 

2 

1 

3 

658 

1715 

1390 

892 

1 

40 

C 

2 

2 

1 

1226 

4045 

3281 

1325 

1 

40 

C 

2 

2 

2 

1027 

3455 

2694 

1039 

1 

40 

C 

2 

2 

3 

905 

3116 

2248 

897 

1 

40 

c 

3 

1 

1 

533 

1028 

1255 

433 

1 

40 

c 

3 

1 

2 

496 

983 

1157 

385 

1 

40 

c 

3 

1 

3 

403 

874 

967 

329 

1 

40 

c 

3 

2 

1 

738 

1747 

2086 

391 

I 

40 

c 

3 

2 

2 

625 

1588 

1780 

363 

1 

40 

c 

3 

2 

3 

532 

1549 

1573 

357 

1 

60 

A 

1 

1 

1 

1247 

3085 

2273 

1986 

1 

60 

A 

1 

1 

2 

1545 

3650 

2780 

2506 

1 

60 

A 

1 

1 

3 

1954 

4437 

3396 

3123 

1 

60 

A 

1 

2 

1 

1616 

5330 

3894 

1910 

1 

60 

A 

1 

2 

2 

2050 

6186 

4719 

2461 

1 

60 

A 

1 

2 

3 

2532 

7232 

5656 

3165 

1 

60 

A 

2 

1 

1 

975 

2216 

1951 

1349 

1 

60 

A 

2 

1 

2 

1158 

2538 

2251 

1629 

Table  31 — continued , 
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CM    N    n^/nj   D      e  S^rSj 


B 


DT 


w 


w 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 


2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1519 
1355 
1616 
1980 
615 
692 
824 
778 
944 
1189 
1298 
1204 
1229 
1752 
1733 
1693 
1024 
1018 
967 
1381 
1302 


3119 
4080 
4619 
5342 
957 
1064 
1153 
1583 
1840 
2097 
3279 
3025 
3079 
5618 
5695 
5513 
2287 
2233 
2283 
4099 
4016 


2891 
3357 
3950 
4748 
1424 
1611 
1903 
2190 
2650 
3227 
2364 
2260 
2295 
4081 
4121 
3950 
2045 
1998 
1943 
3424 
3340 


2091 
1356 
1575 
1981 
445 
508 
550 
382 
468 
532 
2182 
2031 
2026 
2183 
2148 
2075 
1458 
1375 
1377 
1297 
1335 


Table  31 — continued, 


141 


CM    N     n^/nj   D      t  Z^:Z2 


B 


DT 


w 


RT 


S(C) 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


B 
B 
B 
B 
B 
B 
B 
C 
C 
C 

c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 


2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1341 
605 
590 
622 
853 
812 
809 
1248 
1075 
918 
1697 
1409 
1210 
970 
891 
707 
1376 
1168 
975 
574 
484 


4042 
882 
896 
996 
1615 
1599 
1621 
3069 
2718 
2390 
5485 
4828 
4221 
2245 
2026 
1756 
4112 
3705 
3180 
952 
877 


3357 
1380 
1313 
1383 
2304 
2298 
2289 
2416 
2016 
1765 
3975 
3428 
2952 
1996 
1791 
1526 
3356 
3009 
2456 
1336 
1197 


1294 
407 
405 
488 
382 
393 
384 
2002 
1717 
1445 
2102 
1694 
1394 
1338 
1205 
940 
1345 
1200 
942 
428 
403 


Table  31--continued , 
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CM    N     n^/Hj   D      e  S^iSj 


B 


DT 


w 


RT 


S(C) 


1 
1 
1 
1 

2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


60 
60 
60 
60 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


C 
C 
C 
C 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 


3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


421 
757 
673 
566 
1022 
1226 
1583 
841 
984 
1219 
811 
941 
1183 
680 
789 
941 
478 
529 
687 
431 
448 


862 
1656 
1526 
1447 
2713 
3132 
3715 
2432 
2867 
3331 
2000 
2138 
2718 
1856 
2109 
2352 
885 
919 
1120 
945 
974 


1043 
2243 
1963 
1687 
1953 
2261 
2786 
1870 
2188 
2554 
1638 
1885 
2405 
1619 
1936 
2172 
1150 
1299 
1630 
1216 
1316 


368 

402 
325 
317 
1561 
1896 
2341 
1364 
1679 
2048 
1056 
1217 
1590 
959 
1150 
1318 
345 
380 
444 
380 
397 


Table  31 — continued . 
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CM    N    n^/nj   D      e  S^iSj 


B 


DT 


w 


w 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


A 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 


3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


509 
1049 
1076 
1032 
841 
832 
850 
843 
843 
810 
664 
637 
646 
476 
482 
489 
403 
393 
416 
1063 
862 


997 
2699 
2758 
2750 
2545 
2491 
2561 
1972 
2026 
1990 
1844 
1836 
1821 
813 
830 
882 
863 
817 
884 
2755 
2266 


1490 
1883 
1854 
1888 
1823 
1818 
1852 
1614 
1611 
1706 
1592 
1583 
1580 
1132 
1193 
1148 
1108 
1067 
1165 
1957 
1606 


433 
1637 
1646 
1585 
1449 
1419 
1428 
1110 
1137 
1057 
1007 
958 
959 
324 
319 
338 
354 
340 
346 
1620 
1268 


Table  31 — continued, 


144 


CM    N    n^/nj    D      e  1.^:1,^ 


B 


DT 


w 


w, 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
60 
60 
60 
60 
60 


C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
A 
A 
A 
A 
A 


1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


756 
834 
734 
645 
816 
706 
595 
636 
578 
512 
490 
417 
350 
398 
365 
321 
1098 
1347 
1626 
867 
1028 


2013 
2459 
2187 
2002 
2032 
1768 
1608 
1848 
1624 
1552 
891 
891 
742 
877 
833 
805 
2718 
3209 
3745 
2446 
2871 


1352 
1889 
1669 
1488 
1684 
1532 
1306 
1582 
1465 
1241 
1129 
1057 

893 
1122 
1050 

924 
2004 
2394 
2893 
1899 
2184 


1044 
1370 
1152 
979 
1035 
875 
725 
938 
768 
714 
348 
307 
252 
347 
307 
276 
1609 
2007 
2381 
1404 
1704 


Table  31 — continued, 
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CM    N    n^/nj   D      e  S^cSj 


B 


DT 


w 


w, 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
B 
B 
B 
B 
B 
B 
B 
B 


1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1304 
866 
1081 
1290 
694 
806 
1002 
495 
566 
739 
418 
473 
586 
1135 
1138 
1077 
895 
846 
867 
845 
882 


3443 
2018 
2348 
2656 
1907 
2076 
2434 
869 
904 
1100 
858 
912 
1011 
2744 
2795 
2684 
2604 
2522 
2516 
1909 
1980 


2690 
1722 
2126 
2444 
1721 
1952 
2285 
1196 
1416 
1719 
1289 
1383 
1629 
2032 
2077 
1968 
1990 
1932 
1904 
1723 
1750 


2150 
1086 
1371 
1609 
1005 
1162 
1393 
348 
383 
459 
344 
362 
457 
1719 
1717 
1626 
1527 
1465 
1438 
1080 
1130 
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CM    N     n^/nj   D      e  S^rSj 


B 


DT 


w 


w 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 
C 
C 
C 
C 

c 
c 


2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


865 
652 
668 
685 
519 
528 
563 
443 
413 
401 
1104 
940 
816 
867 
760 
625 
829 
728 
656 
694 
583 


1981 
1764 
1776 
1855 
759 
759 
858 
795 
784 
801 
2756 
2421 
2147 
2382 
2208 
1940 
1919 
1716 
1601 
1869 
1647 


1747 

1625 
1695 
1673 
1222 
1224 
1252 
1182 
1273 
1129 
2005 
1803 
1529 
1830 
1738 
1488 
1670 
1510 
1359 
1737 
1434 


1124 
968 
978 
981 
315 
311 
347 
347 
318 
335 
1637 
1365 
1193 
1386 
1228 
1028 
1084 
934 
828 
999 
834 
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CM    N     n^/ng   D      e  S^rSj 


B 


DT 


W 


w, 


RT 


S(C) 


2 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


60 
60 
60 
60 
60 
60 
60 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


C 
C 
C 
C 
C 
C 
C 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 


2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


516 
494 
420 
389 
434 
349 
342 
826 
1028 
1272 
759 
887 
1126 
666 
770 
973 
631 
714 
904 
392 
444 


1538 
856 
777 
765 
892 
782 
770 
1553 
1860 
2202 
1720 
1965 
2393 
1180 
1295 
1574 
1280 
1463 
1740 
545 
604 


1314 
1232 
1005 

911 
1309 
1062 

999 
1178 
1348 
1659 
1329 
1475 
1757 

986 
1126 
1393 
1094 
1275 
1508 

709 

779 


734 
358 
281 
259 
372 
317 
274 
1295 
1591 
1973 
1186 
1439 
1818 
895 
1063 
1341 
828 
986 
1222 
287 
342 
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CM    N    n^/nj   D      e  E^rSj 


B 


DT 


w 


w, 


RT 


S(C) 


3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 


A 
A 
A 
A 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 
B 


3 
3 
3 
3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


554 
387 
397 
500 
840 
866 
841 
778 
739 
714 
682 
663 
699 
634 
592 
611 
385 
374 
425 
361 
349 


695 
628 
643 
734 
1582 
1589 
1594 
1718 
1700 
1648 
1177 
1135 
1192 
1309 
1250 
1284 
481 
509 
567 
561 
549 


941 
782 
865 
1008 
1088 
1138 
1137 
1210 
1162 
1174 
1005 
934 
992 
1100 
1064 
1061 
665 
648 
726 
767 
759 


411 
312 
322 
377 
1393 
1388 
1348 
1266 
1200 
1120 
973 
937 
936 
893 
827 
852 
291 
292 
313 
294 
285 


Table  31 — continued, 
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CM     N     n^/Hj    D       e  2^:22 


B 


DT 


w 


w 


RT 


S(C) 


3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
40 
60 
60 


B 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
C 
A 
A 


3 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 


2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


350 
864 
750 
625 
776 
675 
583 
669 
601 
488 
648 
542 
463 
384 
342 
289 
360 
320 
302 
934 
1060 


588 

1593 
1399 
1258 
1737 
1549 
1370 
1158 
1083 
936 
1293 
1158 
1031 
553 
515 
510 
609 
559 
581 
1651 
1826 


721 
1185 
1029 
929 
1305 
1149 
1010 
1012 
926 
768 
1104 
957 
852 
683 
625 
574 
736 
684 
626 
1204 
1337 


290 

1305 
1094 
909 
1204 
1036 
877 
900 
780 
632 
844 
677 
582 
291 
249 
237 
296 
253 
242 
1423 
1622 


Table  31 — continued. 
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CM    N    n.,/n2   D      e  Z^-.Z^ 


B 


DT 


w 


w, 


RT 


S(C) 


3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 


60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 
60 


A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
A 
B 
B 
B 
B 
B 


1 
1 
1 
1 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
1 
1 
1 
1 
1 


1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 
2 
1 
1 
1 
2 
2 


3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 
3 
1 
2 


1270 
781 
961 

1137 
674 
823 

1016 
656 
713 
937 
380 
460 
600 
400 
428 
535 
899 
901 
865 
803 
810 


2161 
1638 
1987 
2320 
1119 
1292 
1537 
1259 
1388 
1692 
487 
565 
642 
566 
576 
671 
1553 
1602 
1561 
1740 
1746 


1676 
1276 
1492 
1743 
1026 
1198 
1424 
1153 
1237 
1562 
654 
823 
984 
807 
878 
1082 
1139 
1193 
1145 
1254 
1251 


1962 
1162 
1486 
1778 
922 
1093 
1333 
855 
959 
1221 
281 
341 
411 
296 
300 
385 
1387 
1414 
1359 
1262 
1298 


1 
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Table  31 — continued. 


CM 

N 

n^/nj 

D 

c 

^1  *  ^2 

W 

S(C) 

3 

60 

B 

1 

2 

3 

795 

1719 

1252 

1221 

3 

60 

B 

2 

1 

1 

704 

1111 

1006 

942 

3 

60 

B 

2 

1 

2 

693 

1134 

1009 

966 

3 

60 

B 

2 

1 

3 

708 

1123 

986 

935 

3 

60 

B 

2 

2 

1 

636 

1234 

1096 

844 

3 

60 

B 

2 

2 

2 

654 

1308 

1118 

881 

3 

60 

B 

2 

2 

3 

653 

1302 

1104 

868 

3 

60 

B 

3 

1 

1 

451 

490 

724 

311 

3 

60 

B 

3 

1 

2 

415 

471 

703 

280 

3 

60 

B 

3 

1 

3 

416 

496 

722 

290 

3 

60 

B 

3 

2 

1 

399 

530 

822 

288 

3 

60 

B 

3 

2 

2 

410 

550 

818 

290 

3 

60 

B 

3 

2 

3 

407 

567 

812 

288 

3 

60 

C 

1 

1 

1 

909 

1581 

1205 

1377 

3 

60 

C 

1 

1 

2 

762 

1386 

1015 

1158 

3 

60 

C 

1 

1 

3 

650 

1217 

912 

955 

3 

60 

C 

1 

2 

1 

758 

1705 

1269 

1183 

3 

60 

C 

1 

2 

2 

694 

1509 

1165 

1037 

3 

60 

C 

1 

2 

3 

590 

1348 

1004 

882 

3 

60 

C 

2 

1 

1 

730 

1140 

1067 

940 

3 

60 

C 

2 

1 

2 

580 

1017 

887 

781 

i 

\ 
I 
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Table  31 — continued . 


CM  N  n^/n2  D  e  2^:22  B^j  W  Wp^  S(C) 

3  60  C  2  1  3  513  923  803  672 

3  60  C  2  2  1  622  1235  1116  827 

3  60  C  2  2  2  537  1128  1000  713 

3  60  C  2  2  3  449  975  849  578 

3  60  C  3  1  1  411  507  712  301 

3  60  C  3  1  2  364  483  627  256 

3  60  C  3  1  3  319  476  560  234 

3  60  C  3  2  1  387  550  811  274 

3  60  C  3  2  2  349  552  734  282 

3  60  C  3  2  3  292  536  615  253 
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